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ABSTRACT 


Naval  Aviation  is  being  required  to  defend  the  nation  in  transformational  ways, 
with  constrained  resources.  To  support  these  efforts,  the  Naval  Aviation  Enterprise 
(NAE)  was  created  to  improve  the  life-cycle  management  of  naval  aviation  warfare 
systems.  To  meet  the  demands  of  this  vision,  the  NAE  has  created  a  program  called 
AIR Speed  to  deliver  the  efficiency  gains  of  continuous  process  improvement  (CPI).  NAE 
leadership  seeks  a  common  self-assessment  tool  to  measure  how  well  AIRSpeed  has  been 
implemented,  including  possible  areas  for  improvement.  This  thesis  studies  the  origins  of 
continuous  process  improvement,  the  value  of  assessment,  and  current  assessment 
methodologies.  Key  concepts  are  cited  for  the  use  of  organizational  assessment  tools. 
The  objectives  are  an  enhanced  body  of  knowledge  for  enterprise  assessment,  to  provide 
a  comparison  of  several  approaches,  and  to  recommend  a  tool  for  NAE  AIRSpeed.  The 
tools  to  be  considered  are  the  Department  of  Defense  CPI  Transformation  Guidebook 
(DoD  CPITG),  the  Navy's  Performance  Excellence  Guidebook  (NPEG),  the  Lean 
Aerospace  Initiative  (LAI)  Government  Lean  Enterprise  Self-Assessment  Tool 
(GLESAT),  and  the  NAVAIR  Alignment  Assessment  Tool  (NAAT).  Research  revealed 
that  three  of  the  four  tools  (DoD  CPITG,  NPEG,  and  GLESAT)  could  efficiently  measure 
AIR Speed  maturity,  and  recommendations  were  made  that  a  performance  management 
framework  be  established  by  the  NAE  prior  to  implementation  of  any  formal  assessment 
process.  Also,  NAE  can  leverage  the  work  being  done  by  the  LAI  by  using  their  lean 
enterprise  toolset  and  participating  in  the  yearly  plenary  conference.  Further  research  by 
NAE  graduate  students  in  the  MIT  LAI  enterprise  transformation  areas  can  also  advance 
and  leverage  knowledge.  The  maturity  level  of  NAE  AIRSpeed  could  be  baselined  using 
the  LAI  tool  in  those  areas  where  lean  principles  have  been  implemented  over  a  period  of 
years. 
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EXECUTIVE  SUMMARY 


This  joint  thesis  compares  and  contrasts  four  organizational  self-assessment  tools 
that  are  being  considered  to  assess  the  successful  implementation  of  the  Naval  Aviation 
Enterprise  (NAE)  AIR Speed  continuous  improvement  approach.  NAE  is  a  warfighting 
partnership  that  improves  the  life-cycle  management  of  naval  aviation  warfare  systems 
where  interdependent  decisions  affecting  multiple  commands  are  resolved  on  an 
Enterprise-wide  basis.  NAE  leadership  seeks  a  common  self-assessment  tool  to  measure 
how  well  MKSpeed  has  been  implemented,  including  possible  areas  for  improvement. 

The  NAE  is  concerned  with  improving  processes  that  drive  readiness  and  cost.  In 
recent  years,  a  continuous  improvement  effort  called  “AIR Speed"  has  been  implemented 
within  the  NAE  in  three  areas:  (a)  the  production  line  in  the  aviation  depots,  (b)  aircraft 
squadron  and  intermediate  maintenance,  and  (c)  transactional  and  non-production  service 
environments.  Four  relevant  organizational  self-assessment  tools  and  methodologies 
have  emerged  that  can  measure  the  productivity  gains  and  maturity  of  MKSpeed 
continuous  improvement  efforts. 

The  thesis  objectives  are  to  enhance  the  body  of  knowledge  on  enterprise 
assessment,  compare  several  approaches,  and  to  recommend  a  tool  to  continually  assess 
NAE's  MKSpeed  progress.  The  tools  to  be  considered  are  the  Department  of  Defense 
(DoD)  Continuous  Process  Improvement  Transformation  Guidebook  (DoD  CPITG),  the 
Navy's  Performance  Excellence  Guidebook  (NPEG),  the  Massachusetts  Institute  of 
Technology  Lean  Aerospace  Initiative  Government  Lean  Enterprise  Self-Assessment 
Tool  (GLESAT),  and  the  Naval  Air  Systems  Command  Alignment  Assessment  Tool 
(NAAT). 


XIX 


The  primary  research  question  for  this  thesis  was: 

Which  self-assessment  tool(s)  and  methodology  best  captures  the  crucial  aspect  of 

measuring  progress  of  continuous  process  improvement  within  the  NAE? 

The  subsequent  questions  were: 

1.  How  can  the  NAE  leverage  work  being  accomplished  by  the  MIT  Lean 
Aerospace  Initiative  (LAI)? 

2.  How  can  our  "learning"  be  translated  to  provide  value  to  the  MIT  LAI 
research  in  lean  enterprise  transformation? 

3.  Can  a  single  tool  be  scaled  for  use  across  the  NAE? 

The  thesis  studies  the  origins  of  continuous  process  improvement,  the  value  of 
assessment,  and  current  enterprise  assessment  methodologies  in  industry.  Key  concepts 
are  cited  that  require  consideration  when  using  tools  for  performing  organizational 
assessments.  Each  of  the  four  tools  being  researched  are  defined  and  discussed.  Each 
tool  has  been  analyzed  against  the  key  concepts  and  foundational  assessment 
methodologies,  and  results  and  analysis  of  interviews  with  industry'  and  government 
personnel  are  provided. 

We  conclude  and  recommend  that  three  of  the  four  tools  under  study  could 
effectively  be  used  to  measure  AIR Speed  maturity.  Follow-on  research  is  recommended 
for  measuring  AIR Speed  maturity  in  an  area  that  has  used  lean  for  a  number  of  years, 
using  one  of  the  tools;  investigate  whether  the  principles  of  the  High  Performance 
Organization  can  provide  value  to  the  AIR Speed  effort;  and  apply  AIR Speed  tools  to  the 
budget  process. 
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I. 


INTRODUCTION 


A.  BACKGROUND 

We  shall  not  cease  from  exploration  and  the  end  of  all  our  exploring  will 

be  to  arrive  where  we  started  and  know  the  place  for  the  first  time. 

-  T.S.  Eliot 

We  are  currently  living  in  a  time  that  is  alternately  confusing  and  exhilarating. 
We  are  a  nation  fighting  a  war  on  terror,  and  at  the  same  time  attempting  to  manage  our 
position  as  the  world's  superpower.  The  world  is  more  electronically  connected  than 
ever  before,  and  speed  seems  an  essential  ingredient  to  success.  We  live  in  an  age  of 
change;  however,  we  “are  also  living  through  a  change  of  age.  In  the  decade  ahead,  the 
collective  choices  and  actions  of  people,  businesses,  organizations,  and  governments 
everywhere  will  likely  define  and  shape  global  civilization  for  the  next  generation  and 
beyond"  (Kelly,  2006). 

The  U.  S.  Navy,  including  Naval  Aviation,  is  being  required  to  defend  the  nation 
in  different  ways.  The  overall  mandate  from  defense  executives  is  to  transform 
individuals  and  organizations  from  doing  “business  as  usual”  to  “business  un-usual”  and 
to  create  positive  change  for  defense  business  processes.  The  Chief  of  Naval  Operations 
has  defined  the  mission  as  “a  major  sea  change  from  thinking  of  ourselves  as  a 
conglomerate  of  independent  commands  focused  on  individual  output  to  one  of  an 
enterprise  focused  on  collective  output”  (Chief  of  Naval  Operations  (CNO),  2006). 

In  2005,  Admiral  Vern  Clark,  then  Chief  of  Naval  Operations,  defined  a  strategy 
for  the  Navy's  future  called  “Sea  Power  21”  (SP21)  to  integrate  weapons,  networks,  and 
platforms.  SP21  has  three  core  parts:  Sea  Shield,  Sea  Enterprise,  and  Sea  Warrior.  Sea 
Enterprise  is  the  resource  enabler  for  SP21,  and  provides  the  challenge  and  the  business 
process  change  to  recover  resources  (funding,  people,  and  facilities)  for  recapitalization. 
The  Navy  plans  to  do  this  by  “being  bold  and  innovative  in  our  business  processes... by 
embracing  change.  We  are  trying  to  build  a  culture  where  people  feel  it  is  leadership's 
role  to  come  to  work  every  day  looking  for  new  and  better  ways  to  be  more  efficient  and 
productive”  (Kowba,  2003).  Sea  Enterprise  is  about  balancing  Naval  priorities. 


: 


delivering  the  right  force  with  the  right  readiness  at  the  right  cost,  including  challenging 
assumptions,  encouraging  innovation,  maximizing  productivity,  managing  by  metrics, 
and  driving  execution  to  accomplish  the  mission. 

I.  Naval  Aviation  Enterprise  (NAE) 

The  Naval  Aviation  Enterprise  (NAE)  was  formed  three  years  ago  to  implement 
SP21.  The  NAE  is  a  warfighting  partnership  where  interdependent  issues  affecting 
multiple  commands  are  resolved  on  an  enterprise-wide  basis.  The  vision  of  the  NAE  is 
”to  deliver  the  right  force,  with  the  right  readiness,  at  the  right  cost,  at  the  right  time  - 
today,  and  in  the  future.”  (htln:/,'w'ww.cnaf.navv.mil''nae)  This  construct  is  portrayed  as  a 
revolution  within  the  Navy,  where  individual  organizations  attempt  to  adopt  a  corporate 
model  to  better  control  costs,  improve  communications,  identify  redundancies,  and  drive 
waste  out  of  support  and  operational  processes.  The  NAE  is  meant  to  encourage  inter¬ 
agency  and  inter-service  integration,  stimulate  a  culture  of  innovation  and  productivity, 
and  facilitate  change  when  change  is  needed  to  advance  and  improve  as  an  organization. 

NAE  has  created  a  single  Fleet-driven  metric  to  measure  efficiency  and 
effectiveness:  "Aircraft  and  carriers  ready  for  tasking  at  reduced  cost.” 

(http://www.cnaf.navy.mil/nae)  This  metric  is  the  standard  against  which  NAE  measures 
the  ability  to  deliver  products  tied  to  Fleet-driven  demand,  including  buying  the  right 
amount  of  superior  equipment,  reducing  cycle  time  (less  maintenance  time  for  aircraft 
and  components),  and  implementing  process  efficiencies. 

The  processes  that  deliver  Naval  Aviation  readiness  and  incur  costs  are 
interwoven  among  several  commands,  including  Commander.  Naval  Air  Forces  (CNAF). 
Naval  Air  Systems  Command  (NAVAIR),  Naval  Sea  Systems  Command  (NAVSEA). 
Naval  Supply  Systems  Command  (NAVSUP),  and  the  Naval  inventory  Control  Point 
(NAVICP).  Recently,  the  Navy  Education  and  Training  Command  and  Chief  of  Naval 
Air  Training  w'ere  aligned  under  Naval  Air  Forces  so  that  all  aircraft  operations  and 
training  are  functionally  aligned  under  one  command.  Figure  1  depicts  the  NAE 
construct. 

In  this  new  triad  structure,  the  ultimate  customer  is  the  Fleet,  which  refers  to  the 
w'arllghting  force.  The  Fleet  provides  the  "demand"  or  requirements  to  the  triad.  A  single 
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leader  directs  and  monitors  requirements:  CNAF.  commonly  known  as  the  “Air  Boss." 
Fleet  requirements  are  gathered  and  funded  through  the  Office  of  the  Chief  of  Naval 
Operations  (OPNAV),  and  then  the  various  Systems  Commands,  such  as  NAVAIR. 
execute  the  requirements. 


Figure  1.  Naval  Aviation  Enterprise  Construct  (From:  Moore.  2006) 

2.  NAVAIR  Continuous  Process  Improvement 

In  1999.  NAVAIR  embarked  on  a  continuous  process  improvement  (CPI) 
initiative  called  “AIR Speed."  The  intended  goal  of  AIRSpeed  is  to  decrease  cycle  time 
and  identify  and  eliminate  non-value  added  work  and  inefficiencies.  With  the  hopes  of 
improving  quality  of  service  to  the  Fleet,  another  goal  of  AIR Speed  is  to  identify  and 
reduce  variation  in  products  and  services.  MRSpeed  is  meant  to  empower  employees 
with  the  right  tools  to  control  and  modify'  their  daily  processes  to  eliminate  waste,  reduce 
cycle  time  and  improve  quality  of  work.  MRSpeed  incorporates  the  concepts  and  tools  of 
Lean,  Six  Sigma,  and  Theory  of  Constraints  to  achieve  dramatic  results  in  the  enterprise 
(Moore.  2006). 

MRSpeed,  until  recently,  has  been  four  separate  efforts:  Depot  AIR Speed,  which 
concentrates  on  improvements  in  the  production  process  areas  of  the  Naval  Aviation 
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Depots;  Enterprise  MKSpeed,  which  focuses  efforts  in  the  Fleet-wide  repair  sites  and 
processes;  NAVAIR  AIR Speed,  focusing  on  the  corporate  and  competency  processes 
(transactional  and  service-oriented  areas  of  the  organization);  and  in  2005,  NAVICP, 
which  also  focuses  on  corporate  and  competency  processes.  To  foster  enterprise 
behavior,  the  four  individual  AIRS peed  projects  are  merging  into  a  single  AIRSpeed 
initiative,  managed  by  the  NAE.  The  single  NAE  Fleet-driven  metric  of  "aircraft  and 
carriers  ready  for  tasking  at  reduced  cost... today  and  in  the  future"  will  apply  to  the 
single  merged  AIRSpeed  Program. 

B.  PURPOSE 

The  purpose  of  this  joint  thesis  is  to  compare  and  contrast  four  organizational 
self-assessment  tools  to  determine  NAE's  progress  in  implementing  the  AIRSpeed 
continuous  improvement  approach.  The  NAE  is  concerned  with  improving  processes  that 
drive  readiness  and  cost. 

The  Value  of  Assessment 

Many  organizations  formulate  strategy,  however,  deploying  and  implementing  a 
strategy  is  a  much  more  difficult  task  (Baldrige  National  Quality  Program,  2005).  Two 
key  implementation  questions  that  need  to  be  answered  are,  “are  we  making  progress," 
and  “how  do  we  know?"  The  NAE  leadership  should  know  that  organizational  leaders, 
managers  and  working  level  personnel  understand  the  vision  and  mission  for  AIRS/;?*.'*/ 
implementation  and  that  processes  and  improvements  are  occurring  throughout  the 
organization.  NAE  leaders  should  also  know  the  extent  to  which  vertical  and  horizontal 
communications  are  effective,  i.e.,  understood  and  acted-upon.  Assessment  tools  can 
help  provide  senior  executives  a  view  of  the  continuity  of  their  plans  with  the  actions  of 
their  employees.  One  objective  of  this  study  is  to  assist  leaders  and  managers  to  better 
understand  implementation  problems  and  challenges,  and  to  provide  recommendations 
for  improvements. 

Building  on  a  body  of  knowledge  from  key  concepts  and  organizations  involved 
in  enterprise  assessment,  four  self-assessment  tools  from  the  Department  of  Defense 
(DoD),  industry',  and  academia  are  examined.  The  tools  include  the  DoD  CPI 
Transformation  Guidebook  (DoD  CPITG),  the  Navy's  Performance  Excellence 
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Guidebook  (NPEG),  the  Massachusetts  Institute  of  Technology  (MIT)  LAI’s  Government 
Lean  Enterprise  Self-Assessment  Tool  (GLESAT),  and  the  NAVAIR  Alignment 
Assessment  Tool  (NAAT),  based  on  the  survey  tool  from  GENESYS.  Their  structures 
and  applicability  to  the  NAE  are  compared,  and  conclusions  and  recommendations  are 
drawn  regarding  the  maturity  and  future  assessments  of  ongoing  improvement  efforts. 

C.  RESEARCH  QUESTIONS 

A  key  element  of  enterprise  transformation  is  to  understand  the  organization’s 
current  and  desired  future  state  (Murman,  Allen,  Bozdogan,  Cutcher-Gershenfeld, 
McManus,  Nightingale,  ct  al,  2002).  An  initial  effort  is  to  understand  the  principles  and 
methods  for  describing  an  “as  is”  enterprise,  to  ensure  a  baseline  or  starting  point  for 
analysis.  Then,  a  desired  future  state  is  depicted  to  ascertain  the  extent  of  real  and/or 
perceived  gaps. 

Our  primary  research  question  was: 

Which  self-assessment  tool(s)  and  methodology  best  captures  the  crucial  aspect  of 

measuring  progress  of  continuous  process  improvement  within  the  NAE? 

Subsidiary'  research  questions  that  we  address  are: 

•  How  can  the  NAE  leverage  work  being  accomplished  by  the  MIT  Lean 
Aerospace  Initiative  (LAI)? 

•  How  can  our  "learning"  be  translated  to  provide  value  to  the  MIT  LAI 
research  in  lean  enterprise  transformation? 

•  Can  a  single  tool  be  scaled  for  use  across  the  NAE? 

D.  BENEFITS  OF  STUDY 

Assessment  tools  can  provide  leaders  and  managers  methods  for  determining  the 
relative  fit  or  alignment  between  the  direction  they  set  and  the  actual  accomplishment  of 
goals  and  objectives  by  the  workforce.  Various  tools  can  also  be  used  to  focus 
improvements  and  communication  efforts,  including  funding  and  other  implementation 
issues.  This  thesis  provides  the  NAE  with  a  recommended  assessment  tool  approach, 
including  strengths,  weaknesses  and  areas  for  improvements.  The  NAE  leadership  will 
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have  a  depiction  of  a  desired  future  state  and  a  method  to  quantify  the  progress  being 
made  towards  that  future. 

E.  SCOPE  AND  LIMITATIONS 

1.  Scope 

The  scope  of  the  thesis  includes  available  assessment  tools  focused  on  internal 
assessment  of  government  and  industry  organizations.  Four  organizational  self- 
assessment  tools  are  examined: 

•  DOD  CPI  Transformation  Guidebook 

•  Navy  Performance  Excellence  Guidebook 

•  MIT  LAI  GLESAT 

•  NAVAIR  Alignment  Assessment  Tool 

The  tools  are  evaluated  for  their  fit  and  relevance  to  the  NAE,  and  the  evaluation 
ensures  that  recommendations  are  scalable  within  the  enterprise. 

2.  Limitations 

The  George  Group's  Lean  Six  Sigma  maturity  assessment  was  going  to  be 
assessed,  however,  the  George  Group  is  under  contract  to  NAVAIR  to  assist  with 
AIR Speed  training  and  implementation,  and  aspects  of  their  assessment  tool  are 
proprietary  and  therefore  could  not  be  disclosed. 

F.  METHODOLOGY 

The  thesis  assesses  organizational  self-assessment  tools  through  a  literature 
foundation  of  industry  and  academic  research  in  the  subject  area.  Literary  reviews  are 
supplemented  with  (10)  semi-structured  interviews  conducted  with  individuals  in 
academia,  industry,  and  government.  The  methodology  details  are  summarized  below, 
and  are  illustrated  in  Figure  2: 

1.  We  performed  literature  reviews  for  assessment  criteria  from  Malcolm 
Baldrige,  International  Standards  Organization  (ISO),  Software 
Engineering  Institute's  (SEI)  Capability  Maturity  Model  Integration 
(CMMI),  and  MIT's  LAI  to  establish  a  baseline  of  knowledge  for 
enterprise  assessment. 
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2.  We  interviewed  points  of  contacts  at  major  aerospace  corporations.  Naval 
activities,  and  industry  users  to  establish  context  and  requirements  for 
organizational  assessment.  Interview  questions  were  created  and 
standardized  for  each  interview.  The  knowledge  gained  from  the 
interviews  provided  a  solid  foundation  for  an  objective  review  of  the 
assessment  tools. 

3.  We  reviewed  each  assessment  tool  candidate  and  associated  guidance. 

4.  We  researched  case  studies,  as  available,  for  tool  applications  in  industry 
and  government 

5.  We  created  a  matrix  based  on  research  and  interviews  for  comparing  and 
contrasting  the  tools. 

6.  We  made  recommendations  to  our  NAE  sponsor  regarding  a  tool,  or  a 
combination  of  tools,  that  would  best  meet  the  needs  of  NAE  for 
assessment  of  their  continuous  process  improvement  efforts. 


AIRSpeec/  CPI 
Assessment  Tool 


ecommend 


Review  and  analyze: 


DoD  CPI 


Navy  PEG 


LAI  GLESAT 


NAVAIR 

AAT 


Ttt 

Literature  reviews  and  interviews: 

Lean.  Knowledge  Management, 
Change  Management,  Culture 


Investigate  assessment  foundation 
through  literature  reviews: 

CMMI,  Baldrige,  ISO 


Figure  2.  Thesis  Methodology 
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G.  ORGANIZATION  OF  STUDY 

Following  Ibis  introduction.  Chapter  II  reviews  enterprise  assessment  literature 
from  DOD,  academia,  and  industry  to  establish  the  basis  for  organizational  self- 
assessment  of  CPI.  Chapter  III  outlines  the  content  of  the  specific  assessment  tools  being 
studied  for  NAE.  Chapter  IV  presents  comparative  analyses  of  the  tools,  provides  results 
of  interviews,  and  a  summary  of  findings.  Chapter  V  provides  conclusions, 
recommendations,  discussion,  and  proposed  areas  of  further  research. 
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II.  KEY  CONCEPTS  AND  ENTERPRISE  ASSESSMENT 

METHODOLOGIES 

The  road  to  wisdom?  Well,  it's  plain  and  simple  to  express.  Err  and  err 

and  err  again  but  less  and  less  and  less.  -  Piet  Hein  (Kelly,  2006) 

A.  INTRODUCTION 

Why  do  companies  and  government  agencies  adopt  a  continuous  improvement 
approach  and  use  performance  assessment  tools?  This  chapter  provides  a  brief  overview 
of  continuous  improvement,  including  industry  standards  developed,  and  organizational 
assessment  tools  designed  to  measure  the  progress  of  continuous  improvement  efforts. 
The  following  four  assessment  methodologies  are  discussed:  Malcolm  Baldrige  Quality 
Program,  Massachusetts  Institute  of  Technology  (MIT)  Lean  Aerospace  Initiative,  the 
Software  Engineering  Institute's  Capability  Maturity  Matrix  Integration  (CMMI),  and  the 
International  Standards  Organization  (ISO)  9000  quality  standard.  Also  discussed  are  key 
concepts  and  principles  applicable  for  analyzing  the  dimensions  of  enterprise  assessment. 

B.  ORIGINS  OF  CONTINUOUS  IMPROVEMENT 

Continuous  improvement  is  a  derivative  of  the  overall  quality  movement  that  W. 
Edwards  Denting  taught  to  Japanese  companies  after  WWII,  and  which  was  later  adopted 
by  many  American  companies  in  the  1980s  (Walton,  1990).  Concerns  about  quality  had 
early  origins  in  manufacturing,  where  medieval  European  guilds  organized  craftsmen  into 
trade  unions  to  improve  and  coordinate  piecemeal  product  production.  The  factory 
system,  which  began  in  Great  Britain  in  the  mid- 1750s,  also  organized  craftsmen  into 
specialized  areas  and  emphasized  product  inspection,  which  grew  into  the  Industrial 
Revolution  in  the  early  1800s.  By  the  late  19Ih  century,  the  United  States  was  moving 
away  from  the  European  model  to  a  new  “scientific  management"  approach  developed  by 
Frederick  W.  Taylor  (Seddon,  2005).  Taylor  wanted  to  increase  productivity  without 
increasing  the  number  of  skilled  craftsmen,  instead  increasing  their  efficiency.  He 
concentrated  on  productivity  gains,  but  unskilled  labor  was  considered  expendable  and 
that  emphasis  started  to  have  a  negative  effect  on  quality.  Factory  managers  created 
inspection  departments  to  remedy  the  decline  in  quality,  but  it  was  often  more  common  to 
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ask  how  a  defective  product  reached  the  customer,  rather  than  asking  why  it  was  made 
that  way. 


The  American  Society  for  Quality  website  discusses  the  history  of  quality  on  its 
website  (http://www.asq.org/learn-about-quality/history-of-quality/overview/20th- 
eenturv.html).  In  the  early  20'1'  century,  the  element  of  "processes"  started  to  be  included 
in  the  description  of  quality  practices.  A  process  was  defined  as  “a  group  of  activities 
that  takes  an  input,  adds  value  to  it  and  provides  an  output."  Measurement  of  processes 
began  in  the  1920s  using  statistical  quality  control  methods  to  determine  if  products 
could  be  produced  more  uniformly  and  with  fewer  defects.  Quality  became  essential 
during  World  War  II  when  the  United  States  military  needed  bullets  that  were 
consistently  usable  in  rifles  while  being  manufactured  in  various  factories  across  the 
United  States.  Initially,  the  military  performed  100  percent  inspection,  but  because  of 
published  standards  based  on  statistical  quality  control,  they  were  able  to  move  to 
sampling  techniques. 

Following  World  War  11,  W.  Edwards  Deming  and  Joseph  Juran,  finding  no 
American  audience  for  their  theories  on  total  quality,  went  to  Japan  to  assist  in  the 
recovery  of  a  severely  crippled  manufacturing  industry.  Japanese  industry  embraced  and 
acted  upon  total  quality  concepts  and  statistical  process  control,  eventually  achieving 
world-class  success.  Total  quality  began  in  the  United  States  as  a  direct  response  to 
Japan's  improved  manufacturing  processes  and  quality  products,  which  were  now 
directly  competing  with  United  States  industry.  Many  in  this  country  considered  "Total 
Quality  Management"  (TQM)  a  business  fad,  and  even  though  the  term  has  faded  from 
use,  similar  quality  practices  can  be  found  in  many  Fortune  500  companies  today. 

C.  THE  COLD  WAR  AND  WEAPON  SYSTEMS 

Aerospace  manufacturers  were  able  to  continue  their  manufacturing  rate  after 
World  War  11  with  the  advent  of  the  Cold  War.  Waves  of  new  military  aircraft  rolled  off 
assembly  lines.  The  defense  budget  dominated  federal  outlays.  Continuing  past  the 
Vietnam  War,  funding  levels  remained  relatively  high,  fueled  by  a  pervasive  Soviet 
threat.  The  Department  of  Defense  (DoD)  emphasized  performance  over  cost  with  budget 
controls  taking  a  back  seat.  Complexity  in  military'  systems  and  defense  organizations 
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grew,  technology  expanded,  and  demand  increased  for  more  sophisticated  weapon 
systems,  such  as  stealth  aircraft  and  smart  munitions.  Cost  overruns  were  often  handled 
by  expanding  schedules  to  postpone  outlays.  The  time  to  develop  major  weapon  systems 
increased  by  80  percent  during  the  latter  half  of  the  Cold  War  (Murman,  et  al,  2002),  and 
cost  increases  resulted  in  additional  oversight  and  reduced  industry  flexibility. 

Further,  with  the  end  of  the  Cold  War,  the  aerospace  industry  soon  found  itself 
with  excess  capacity  and  a  reduced  demand  market.  The  industry  needed  to  find  ways  to 
survive  and  grow.  Military  systems  aged,  and  replacement  costs  were  often  prohibitively 
high.  DoD  missions  changed  with  the  fall  of  the  Soviet  Union  and  the  Berlin  Wall,  and 
the  aerospace  industry  found  itself  misaligned  with  changing  national  priorities,  new 
market  conditions,  and  the  technological  maturity  of  core  products,  all  of  which 
contributed  to  the  need  for  substantial  improvements.  Industry'  and  DOD  seized  on 
various  tools  and  methods  such  as  downsizing,  outsourcing,  privatizing,  and  mergers  and 
acquisitions,  including  ubiquitous  reorganizations.  Modes  of  thinking  ingrained  during 
the  five  decade  Cold  War  often  acted  as  barriers  to  needed  transformation. 

D.  INDUSTRY  AND  “THE  TOYOTA  WAY” 

Toyota  felt  that  one  way  to  build  Japan's  economy  was  to  "catch  up  to  the 
Americans  in  three  years”  (Murman,  et  al,  2002).  However,  when  Toyota's  engineers 
visited  the  Ford  Motor  Company  in  the  United  States,  they  saw  mass  production  with 
high  volumes,  large  production  lots  and  stored  inventories.  Their  interpretation  of 
effective  production  management  was  different  than  their  Ford  counterparts.  They  saw 
“flow”  and  the  necessity  to  have  flow  match  demand,  which  up  to  then  was  perceived 
differently  in  the  United  States.  The  Toyota  engineers  returned  to  Japan  and  developed  a 
production  system  to  match  their  capabilities  and  economic  realities.  They  created  their 
own  dynamic  processes  with  help  from  Juran  and  Deming:  just-in-time  deliver)',  process 
control,  integrated  process  and  product  design,  and  the  notion  of  quality  not  necessarily 
costing  more.  They  developed  methods  in  response  to  specific  problems,  and  those 
problems  were  framed  the  way  they  thought  about  the  design  and  management  of  work. 
Over  time,  they  created  company  cultures  based  on  sustained  and  applied  learning.  When 
Americans  later  went  to  visit  Toyota  to  see  the  secret  of  their  success,  they  saw  solutions 


applied  to  specific  situations,  safe  and  tidy  organized  workplaces,  product  designs  that 
controlled  the  product  being  designed,  and  workers  and  managers  solving  problems 
together  (Scddon,  2005). 

Many  companies  across  a  number  of  industries  in  the  United  States  began  to 
adopt  so-called  “Lean  principles,"  or  the  “Toyota  way,"  to  improve  and  evaluate 
performance  including  reliance  on  various  assessment  tools. 

E.  LEAN  AND  LEAN  THINKING 

Toyota  took  over  30  years  to  create  their  success,  but  they  are  now  a  worldwide 
leader  in  the  automobile  industry.  Toyota  did  not  coin  the  term  “lean;”  the  term  was 
coined  as  a  result  of  research  done  by  MIT  (Murman,  2002).  In  1993,  MIT  pioneered  the 
concept  of  lean  thinking  and  practice  and  formed  the  Lean  Aerospace  Initiative  (LAI). 

The  word  “lean"  represented  the  ideas  of  economy  of  effort,  minimizing  waste, 
and  working  hand-in-hand  with  suppliers  to  manage  How.  It  means  creating  value  by 
doing  the  right  job,  right.  The  Toyota  culture  was  one  of  learning  how  to  create  value  for 
customers  by  continual  and  steadfast  focus  on  the  above  concepts,  including  learning 
from  failures  (e.g.,  Honda  motorcycles  in  the  U.S.). 

The  MIT  LAI  defines  lean  as  “eliminating  waste  and  creating  value"  (Murman,  et 
al,  2002).  Eliminating  waste  and  creating  value  by  themselves  are  not  sufficient.  LAI 
defines  lean  thinking  as: 

Lean  thinking  is  the  dynamic,  knowledge-driven,  and  customer-focused 

process  through  which  all  people  in  a  defined  enterprise  continuously 

eliminate  waste  with  the  goal  of  creating  value. 

“Waste”  has  many  examples,  especially  in  manufacturing,  but  the  concepts  are 
also  applicable  to  service  organizations.  Activities  that  absorb  resources  but  create  no 
value  arc  examples  of  waste,  i.e.,  mistakes  requiring  fixing  or  rework,  production  of 
unneeded  items  and  inventory  pile-up,  unnecessary  processing  steps,  unnecessary 
movement  of  employees  and  goods,  groups  of  people  in  a  downstream  activity  waiting 
for  upstream  people  to  make  decisions  or  clear  impediments,  and  of  course,  using  energy 
and  resources  that  do  not  contribute  to  customer  needs  and  expectations  (Womack  & 
Jones,  2003). 
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Lean  thinking  provides  a  way  to  specify  value,  align  value-creating  activities  in 
the  most  efficient  sequence,  conduct  activities  without  interruption,  and  to  adapt  to 
internal  and  external  environmental  realities.  Lean  thinking  evidently  provides  a  way  to 
do  more  with  less:  less  human  effort,  less  equipment,  less  time  and  space,  while 
providing  customers  what  they  want.  Lean  should  not  be  confused  with  “Business 
Process  Engineering"  (Hammer  &  Champy,  2003),  which  often  translates  into  cutting 
and  downsizing  personnel  to  obtain  efficiencies  (as  well  as  streamlining  processes). 
Rather,  Lean  is  meant  to  eliminate  waste  and  to  learn  and  internalize  improvements, 
which  can  actually  free  human  energy  to  create  new  work  and  accomplish  existing  work 
smarter  and  more  efficiently. 

F.  WHAT  IS  THE  LEAN  ENTERPRISE? 

LAI  believes  that  a  company  can  fully  realize  lean  value  only  by  adopting  an 
enterprise  perspective.  Murman,  et  al  (2002)  say  "a  lean  enterprise  is  an  integrated  entity 
that  efficiently  creates  value  to  its  multiple  stakeholders  by  employing  lean  principles  and 
practices."  Interdependencies  must  be  addressed  across  enterprise  levels  to  increase  lean 
value.  In  the  lean  enterprise,  customer  value  is  specified  in  a  way  that  does  not  favor  any 
singe  entity’s  definition  in  the  value  stream.  All  of  the  entities  in  the  value  stream 
identify  the  actions  required  to  bring  a  product  from  concept  to  use,  from  requirement  to 
the  battlespace,  or  from  raw  material  into  the  hands  of  the  customer.  Any  actions  that  do 
not  create  value  are  removed  from  the  value  stream.  Actions  that  do  create  value  will 
show  continuous  flow  from  constant  customer  demand.  Results  are  then  analyzed  and 
evaluated  and  the  process  begins  again. 

Womack  and  Jones  (2003)  describe  the  mechanisms  of  the  lean  enterprise  as:  “a 
conference  of  all  the  firms  along  the  stream,  assisted  by  technical  staff...  to  periodically 
conduct  rapid  analyses  and  then  to  take  fast-strike  improvement  actions.”  These  actions 
are  in  their  first  stages  within  the  NAE  with  the  formation  of  the  NAE  War  Council, 
which  has  aligned  the  NAE  entities  to  better  identify  resources  to  get  the  right  things 
done  at  the  right  time. 

A  key  concept  of  a  lean  enterprise  is  that  all  functions  must  become  “transparent" 
for  organizational  learning  to  occur,  i.e.  each  entity  or  function  shares  its  information 
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rather  than  protect  and  hoard  information.  Trust  and  shared  values  are  notions  that  must 
be  carefully  built,  sustained  and  rewarded  across  functions.  Womack  and  Jones  indicate 
that  for  this  to  work,  a  mechanism  for  mutual  verification  must  exist  or  be  created  to 
ensure  that  everyone  abides  by  the  same  criteria,  rules  and  principles.  This  is  what 
organizational  assessment  and  measurement  is  about. 

The  LAI  describes  the  core  principles  of  a  lean  enterprise  as  (Murman,  et  al, 

2002): 

•  Waste  minimization  -  eliminate  non-value  added  activities  and  thereby 
deliver  value  to  the  customer 

•  Responsiveness  to  change  -  a  need  for  agility  in  responding  to  changing 
threats  to  be  able  to  produce  the  product  when  needed 

•  Right  thing  at  right  place,  at  right  time,  and  in  right  quantity 

•  Effective  relationships  within  the  value  stream  -  organizations  function 
more  efficiently  when  there  is  mutual  trust  and  respect,  information 
sharing,  open  and  honest  communication  among  all  stakeholders 

•  Continuous  improvement  -  the  fundamental  pursuit  of  perfection 

•  Quality  from  the  beginning 

A  key  element  of  enterprise  transformation  is  to  understand  the  organization's 
current  and  desired  future  state.  The  “as  is"  and  the  “to  be"  (future)  state  of  the 
organization  must  be  defined  and  analyzed  (Murman,  et  al,  2002).  There  are  a  number  of 
organizational  tools  that  provide  ways  to  assess  performance  and  continuous 
improvement.  Assessment  tools  discussed  in  following  sections  and  chapters  often 
provide  indicators  of  “capability  maturity  assessment,"  including  ways  to  survey  an 
organization  and  evaluate  responses. 
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G.  KEY  CONCEPTS  FOR  ASSESSMENT  TOOLS 

The  following  concepts  are  considered  critical  when  discussing  the  attributes  of 
an  effective  assessment  tool  for  enterprise  use.  They  are  summarized  below  and  more 
fully  addressed  within  this  section. 

•  Alignment,  or  the  extent  to  which  people  and  culture  are  congruent  with 
desired  performance. 

•  The  critical  importance  of  managing  knowledge,  change,  and  culture 

•  Create  value  for  customers  —  The  ultimate  customer  defines  value,  and  it 
means  delivering  what  the  customer  needs  or  wants  (with  an  expectation 
of  quality). 

•  Eliminate  waste,  which  is  any  activity  that  absorbs  resources  but  creates 
no  value. 

•  Have  well-designed  and  mature  processes  --  Processes  allow  an 
organization  to  align  its  business  for  effectiveness,  and  to  leverage 
resources  and  examine  business  trends.  A  focus  on  process  provides  the 
infrastructure  necessary  to  maximize  personnel  and  technology. 

•  Improve  organizational  learning,  performance  practices,  capabilities  and 
results  —  Organizational  learning  includes  not  only  continuous 
improvement  of  existing  approaches,  but  acknowledging  change  and 
adjusting  goals  and  approaches. 

•  Value  employees  and  partners  —  Having  a  diverse  workforce  and  partners 
will  ensure  success  when  their  knowledge,  skills,  creativity,  and 
motivation  are  combined.  Value  can  be  enhanced  by  committing  to 
employee  satisfaction,  development,  and  well-being;  value  is  visible. 
Valuing  partners  and  keeping  both  internal  and  external  relationships 
strong  can  help  identify  complementary  strengths  and  capabilities  to 
address  common  issues. 

•  Create  a  “systems  thinking”  culture  versus  a  command-and-control  culture 
—  Having  a  systems  perspective  means  looking  at  the  organization  as  a 
whole  and  building  on  key  requirements,  including  strategic  objectives 
and  action  plans. 

•  Understanding  the  differences  between  manufacturing  and  production 
organizations  and  service  organizations. 
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1.  Alignment 

Alignment  is  a  crucial  systems  concept  referring  to  alignment  of  organizational 
variables.  In  human  resource  terms,  alignment  means  that  everyone  throughout  the 
organization  -  from  hiring  through  promotions  —  is  focused  and  working  toward  a 
constancy  of  purpose,  i.e.,  recruiting  aligned  with  the  work  technology,  aligned  with  the 
promotion  and  reward  system,  aligned  with  outputs  and  outcomes.  In  sum,  it  is  the  extent 
to  which  important  variables  -  people  skills  and  culture  in  particular  -  are  aligned  or 
congruent  with  desired  performance. 

The  concept  of  alignment  extends  beyond  human  resource  alignment  to  include 
the  extent  to  which  an  organization’s  overall  design  (structure,  technology  and  processes) 
is  aligned  with  its  strategy,  and  how  its  strategy  is  aligned  with  its  external  environment, 
including  stakeholder  needs  and  expectations  (i.e.,  customers  are  an  essential  subset  of 
stakeholders).  Alignment  is  a  theoretical  construct,  a  dynamic  and  ever-changing  ideal 
state  meant  to  increase  the  probability  of  obtaining  superior  performance  and  customer 
satisfaction. 

There  are  vertical  and  horizontal  nominal  dimensions  to  alignment.  The  vertical 
dimension  is  concerned  with  organizational  strategy  and  the  people  that  implement  the 
strategy  through  daily  customer  work.  When  vertical  alignment  exists,  employee  actions 
will  show  that  the  workforce  understands  the  organizational  goals  and  their  role  in 
achieving  them.  Vertical  alignment  energizes  people,  provides  direction,  and  offers 
opportunity  for  involvement  (Labovitz  &  Rosansky,  1997). 

The  horizontal  dimension  involves  the  business  processes  that  create  customer 
value.  Processes  cut  across  the  horizontal  functions  within  an  organization,  and  people 
must  work  together  across  those  functions  to  ensure  alignment  to  customer  needs. 
Horizontal  alignment  ensures  that  the  processes  support  the  customer,  which  should  be 
the  ultimate  goal  of  an  organization.  Those  processes  not  only  create  the  products  and 
services,  but  deliver  them  when  and  where  customers  want. 

Both  vertical  and  horizontal  dimensions  must  be  working  synchronously, 

independently  and  with  each  other.  When  all  of  the  main  organizational  variables  are  in 
relative  alignment,  a  productive  interrelationship  -  the  emerging  culture  -  is  more  likely 
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lo  yield  desired  results.  Appropriate  measures  and  management  interventions  can  then  be 
applied  to  keep  things  working  towards  congruence.  Figure  3  illustrates  the  alignment 
concept. 


Figure  3.  Alignment  Concept  (After  Labovitz  &  Rosansky,  1997) 

Labovitz  &  Rosansky  (1997)  indicate  that  an  organization  can  achieve  a  state  of 
"self-alignment"  by  using  performance  measures  linked  to  awards  and  recognition.  A 
self-aligning  organization  seeks  to  obtain  and  sustain  relative  alignment  while 
anticipating  and  responding  to  environmental  changes.  An  organization  should 
continually  monitor  itself  and  the  outside  environment  for  indications  of  incongruence 
(e.g.,  making  a  product  superseded  by  technology).  While  the  invisible  hand  of  culture 
(i.e.,  enduring  employee  behaviors)  can  guide,  it  can  also  inhibit,  a  transformative 
initiative. 

What  to  measure  becomes  a  challenging  aspect  of  organizational  alignment  and 
assessment.  Key  measures  can  be  developed  and  used,  broad  enough  to  capture  crucial 
aspects,  yet  precise  enough  so  that  employees  understand  what  exactly  is  being  measured 
and  what  their  contribution  is.  Some  measures  identify  various  relationships  among 
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inputs,  design,  outputs  and  outcomes,  and  some  focus  directly  on  customer  feedback.  If 
measurement  systems  themselves  are  not  aligned,  they  can  pull  aspects  of  the 
organization  in  different  directions,  i.e.,  everyone  is  working  very  hard  but  without 
constancy  of  purpose. 

An  example  of  crucial  alignment  concerns  resource  allocation.  If  an 
organizational  design  is  fragmented  or  incongruent,  then  scarce  resources  are  used  in 
unneeded  areas  and  goal  accomplishment  is  threatened.  Resources  in  this  context  include 
people,  money,  equipment,  facilities  and  with  increasing  importance  the  resources  of 
information  and  expertise  (Ancona,  D.,  Kochan,  T.,  Scully,  M.,  Van  Maanen,  J.,  & 
Westney,  D.,  2005). 

Additional  concerns  around  alignment  include  the  following:  To  what  extent 
does  organizational  design  fit  environmental  forces  and  trends?  Are  espoused  strategies 
capable  of  implementation?  Are  senior  executive  leaders  correctly  interpreting  external 
environmental  forces  and  trends,  adjusting  strategies,  and  identifying  desired  results?  Is 
the  organization  getting  needed  inputs  and  efficiently  transforming  those  inputs  into 
customer  satisfaction?  Impediments  to  alignment  are  multiple,  such  as  the  degree  to 
which  employees  understand  and  are  able  to  accomplish  stated  and  implied  goals  and 
tasks.  Other  impediments  include  inadequate  and/or  inaccurate  information,  insufficient 
resources  to  accomplish  assigned  tasks,  and/or  reward  systems  incongruent  with  desired 
behaviors  (Ancona,  et  al,  2005). 

2.  Managing  Knowledge,  Change,  and  Culture  in  the  Lean  Environment 

Recurring  themes  in  the  literature  surrounding  Lean  are  managing  knowledge, 
change,  and  culture.  If  these  broad  concepts  are  not  translated  and  converted  into 
activities  that  can  be  assessed  and  improved,  then  Lean  becomes  another  management  fad 
to  be  discarded  in  the  bins  of  history.  The  next  section  defines  relevant  terms  and  what 
they  mean  to  an  organization  embarked  on  a  lean  journey,  including  the  point  of  this 
thesis  -  assessing  progress  on  the  journey. 

The  term  “knowledge  management”  is  an  abstract  concept  containing  multiple 
definitions.  Individuals,  groups,  and  organizations  not  only  process  knowledge,  but  create 
it.  Seddon  (2005)  states  that  “the  value  of  knowledge  is  in  its  use,  not  its  collection." 
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Knowledge  becomes  more  valuable  the  more  il  is  used  and  combined  with  other 
knowledge. 

Knowledge  includes  explicit  and  tacit  types  of  knowledge.  Explicit  knowledge 
can  be  articulated  in  formal  language  through  statements,  mathematical  expressions, 
manuals,  etc.  This  type  of  knowledge  is  easily  transmitted  throughout  an  organization. 
On  the  other  hand,  Nonaka  and  Takeuchi  (1995)  describe  tacit  knowledge  as  “personal 
knowledge  embedded  in  individual  experience,  and  involves  intangible  factors  such  as 
personal  belief,  perspective,  and  the  value  system.” 

Why  is  understanding  and  recognizing  the  value  of  knowledge  and  its  creation 
important?  Because  of  the  success  of  Japanese  industry'  (a  prime  example  being  Toyota), 
it  is  widely  known  that  what  differentiates  the  Toyota  model  and  contributes  to  its 
success  is  the  way  knowledge  is  received,  processed  and  acted-upon.  Intellectual  capital 
is  now  considered  by  many  businesses  as  being  as  important  as  financial  capital 
(DeLong,  2004).  Japanese  companies  continuously  improve  by  having  a  culture  and 
structure  that  systematically  receives  data  (external  and  internal  to  the  organization)  and 
transforms  it  into  usable  knowledge  that  has  value  in  the  marketplace.  Information 
becomes  knowledge  which  is  shared  and  acted-upon,  including  the  capability  to  create 
and'or  adapt  new  products  (Nonaka  &  Takeuchi,  1995). 

It  is  useful  to  ask  about  the  possible  attributes  of  an  ideal  "knowledge  worker.” 
(S)he  has  a  strong  sense  of  self,  is  well  educated,  wants  to  contribute,  is  mobile,  a  learner, 
is  a  team  player,  is  comfortable  with  technology,  and  shares  knowledge  (Bennet  & 
Bennet,  2004).  Senior  executive  leaders  are  responsible  for  creating  the  climate  and 
organizational  design  that  is  receptive  and  productive  for  employees  to  learn,  i.e.,  a 
learning  organization  entails  structure,  rewards  and  consistent  signals. 

Also  useful  to  consider  are  elements  that  can  impede  an  organization  from 
capturing  and  using  knowledge  (Seddon,  2005).  In  a  traditional,  bureaucratic,  command- 
and-control  structure,  the  "raison  d'etre”  (reason  for  being)  is  stability  and  predictability, 
not  innovation  and  change.  Managers  and  employees  correctly  perceive  a  plethora  of 
impediments  and  constraints,  which  can  filter  out  and  dampen  an  organization's  ability  to 
receive  and  process  knowledge  into  actions,  i.e.,  violate  a  constraint  and  take  a  risk  at 
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your  peril.  In  many  DoD  organizations,  unfortunately,  knowledge  may  be  perceived  as 
commensurate  with  rank  and  hierarchy. 

3.  What's  Important  about  Knowledge  Management  and  Lean? 

Why  is  it  important  to  include  knowledge  management  while  assessing  level  of 
learning  or  Lean  maturity  level?  Stall  with  "waste."  Eliminating  waste  is  paramount  in 
Lean  progress.  Womack  and  Jones  (2003)  have  written  about  this  crucial  aspect,  and 
Bauch  (2004),  in  his  MIT  doctoral  thesis,  described  10  categories  of  waste.  Another  MIT 
doctoral  student  took  Bauch's  work  and  elaborated  on  the  top  three  wastes  in  Bauch's 
list:  over-processing,  rework,  and  information  decay  (Livengood,  2006).  Some  specific 
wastes  of  rework  and  information  decay  are  discussed  as  they  relate  to  managing 
knowledge  along  a  Lean  journey. 

In  any  organization,  creating  new  information  and  processing  existing  information 
are  core  tasks.  When  employees  leave  or  fail  to  document  knowledge,  knowledge  can  be 
lost  or  become  deficient  in  quality.  At  the  same  time,  information  sharing  is  important  in 
an  organization  so  that  knowledge  can  be  created.  If  know  ledge  is  not  routinely  shared,  or 
has  to  be  reinvented,  there  is  a  risk  of  solving  the  same  problems  over  and  over  again. 
On  the  other  hand,  a  frequent  idea  is  to  put  an  information  technology  system  in  place  to 
"manage  knowledge."  However,  if  such  a  system  is  put  in  place  to  manage  knowledge, 
will  the  employees  utilize  it?  When  using  information  systems,  is  more  information  than 
is  needed  being  kept?  Is  it  redundant,  or  are  there  multiple  representations  of  the  same 
ideas?  As  technology  has  grown,  collectively,  our  ability  to  store  vast  quantities  of 
information  has  outpaced  our  ability  to  retrieve,  sort  and  act-upon  that  information.  In 
this  context,  such  accumulation  only  exemplifies  waste. 

a.  Information  Decay 

If  information  processes  are  poorly  executed  or  uncoordinated, 
information  gets  stored  -  information  in  inventory  -  and  is  then  vulnerable  to  outside 
forces  that  can  make  it  defective.  Requirements  and  technical  difficulties  routinely  occur. 
Livengood  (2006)  makes  the  case  that  information  in  inventor)'  decays  at  a  daily  rate  of 
50  percent,  and  each  working  month,  12  percent  of  stored  information  decays. 
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Information  in  inventory  creates  cost,  which  is  calculated  by  a  "rework  ratio"  of  the  time 
spent  on  rework  divided  by  the  time  spent  on  original  work. 

b.  Waiting 

The  waste  of  "waiting”  can  also  affect  the  information  inventory. 
Examples  of  "waiting"  waste  are  when  people  wait  for  answers,  data,  requirements, 
approvals  or  decisions;  information  created  quickly  can  become  obsolete  by  the  time  it  is 
used,  and  rework  may  result  and,  obviously,  waste  occurs  when  people  have  to  wait  for 
other  people  or  machines  to  be  available.  For  example,  this  happens  if  one  person  needs 
a  decision  and  the  decision  maker  is  not  available  or  a  new  employee  comes  on  board  and 
computer  equipment  is  not  available  for  them  to  use. 

c.  Rework 

When  a  task  is  not  done  correctly  the  first  time,  rework  becomes 
necessary.  Why  was  it  done  incorrectly  the  first  time?  Excuses  abound,  but  it  could  be 
the  result  of  fast,  thoughtless,  tasking;  not  having  the  right  documented  information;  or 
knowledge  is  unavailable  to  do  the  task  correctly. 

4.  Culture  and  Change 

Culture  is  both  an  abstraction,  and  a  reference  to  actual  behaviors,  and  the  forces 
derived  from  culture  in  social  and  organizational  situations  are  powerful.  Culture  can  be 
described  in  a  hierarchical  fashion  or  levels,  consisting  of  artifacts,  espoused  beliefs  and 
values,  and  underlying  assumptions  (Schein,  2004). 

Artifacts  are  the  visible  or  perceptive  organizational  structures  and  processes  that 
can  be  seen,  heard,  or  felt  about  the  organization.  Artifacts  can  include  such  things  as 
buildings  and  facilities,  employee  and  manager  dress-codes,  myths,  stories,  rituals  and 
ceremonies.  Although  usually  visible  by  definition,  artifacts  can  be  difficult  or  subtle  to 
decipher.  For  example,  one  can  observe  a  ritual  or  ceremony  but  not  understand  its 
meaning  or  origin.  Espoused  beliefs  or  values  attempt  to  express  the  dominant  culture’s 
intended  values  and/or  belief  system,  which  may  or  may  not  be  reflected  in  actual 
behaviors,  e.g.,  Enron  articulated  all  the  high-sounding  values.  If  beliefs  and  values  are 
only  "espoused  theories,"  then  they  may  well  predict  what  employees  say  in  a  variety  of 
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situations,  but  not  reflect  what  employees  will  actually  do  when  the  values  are  challenged 
in  difficult  or  stressful  situations  (Schein,  2004). 

Basic  underlying  assumptions  are  created  when  a  solution  to  a  problem  works 
repeatedly  and  is  taken  for  granted.  The  degree  of  consensus  about  an  assumption  results 
from  repeated  success  in  implementing  certain  beliefs  and  values.  For  example,  a  basic 
underlying  assumption  rarely  challenged  at  NAVAIR  is  that  supporting  the  warfighter  is 
a  core  doctrine. 

Cultural  subgroups  want  to  perceive  events  as  congruent  with  their  assumptions, 
but  what  if  there  is  a  disconnect  between  assumptions  and  actual  behaviors?  This  is 
where  culture  exerts  its  power  and  influence  -  and  because  it  runs  deep,  it  is  slow  to 
change.  Most  people  tend  to  function  well  in  a  state  of  "cognitive  stability;”  therefore 
any  challenge  or  questioning  of  a  basic  assumption  will  release  anxiety  and  increase 
defensiveness  (Schein,  2004). 

Sustained  top  management  direction  and  support  is  a  prerequisite  for  any  large- 
scale  culture  change  (DeLong,  1994).  To  change  culture,  it  is  appropriate  to  question  old 
beliefs,  values,  and  assumptions  and  assess  them  in  terms  of  their  congruence  with  new- 
organizational  realities.  Leaders  can  offer  a  new  vision  for  employees  to  internalize, 
thereby  stimulating  more  change.  Over  time,  new  assumptions  become  ingrained  and  a 
new  culture  emerges,  i.e.,  a  new  common  history  is  created.  In  sum,  skillful  leaders, 
managers  and  practitioners  recognize  the  powerful  role  culture  performs,  and  intervene  in 
ways  to  shape  and  influence  desired  behaviors,  i.e.,  culture  is  directly  linked  to 
performance. 

5.  Change  Management 

In  Kotter's  (1996)  writings  on  change,  he  cites  eight  steps  that  are  necessary  to 
accomplish  organizational  change.  The  first  step  is  to  create  a  “sense  of  urgency"  for 
change,  i.e.,  a  "burning  platform"  for  change.  The  other  steps  describe  creating  a  strong 
guiding  coalition,  setting  a  clear  vision,  communicating  the  vision  extensively,  removing 
barriers  to  the  vision,  and  creating  short-term  wins  along  the  way,  while  avoiding  the 
tendency  to  declare  victory  too  early  and  neglect  to  anchor  changes  in  the  corporate 
culture  (Kotter,  1996). 
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Possible  consequences  of  disregarding  these  steps  abound:  new  strategies  are 
simply  not  understood,  much  less  implemented;  reengineering  (process  improvement) 
takes  too  long  and  costs  too  much;  downsizing  does  not  get  costs  under  control;  and 
quality  programs  take  on  a  life  of  their  own  but  do  not  result  in  promised  or  hoped-for 
results. 

The  NAE  is  faced  with  hazards  and  opportunities  driven  by  a  fast-changing 
globalized  world  and  a  transforming  DoD.  Many  large,  complex,  bureaucratic 
organizations  are  slow  to  adapt,  especially  in  a  command-and-control  hierarchy  designed 
for  stability  and  predictability.  Without  real  or  perceived  urgency,  employees  default 
towards  status  quo  behaviors,  i.e.,  transformation  cannot  occur  through  mandate  and 
remote  control. 

Employee  “empowerment”  appears  to  be  another  ingredient  relevant  for 
accomplishing  substantial  change.  Critical  information  about  continual  improvement  and 
transformation  can  sit  trapped  in  workers'  minds  waiting  for  approval  to  take  appropriate 
actions.  While  often  used  loosely,  "empowerment"  in  this  context  means  communicating 
a  sensible  vision  to  employees,  removing  barriers  to  alignment,  providing  needed 
training,  and  ensuring  supervisors  support  the  vision  (Kotter,  1996).  Again,  command- 
and-control,  constraint-oriented  organizations  may  not  be  able  or  receptive  to  genuine 
employee  empowerment. 

Seddon  (2005)  says  change  is  a  "problem  of  culture"  and  a  problem  in 
management  thinking.  He  indicates  that  an  organization  must  be  managed  as  a  system, 
and  look  from  an  outside-in  perspective  (the  view  of  the  customer),  that  the  values  of  the 
organization  must  be  based  on  learning,  and  change  managed  as  an  adaptive,  integral 
exercise  in  organizational  growth.  Part  of  systems  thinking  includes  designing 
organizational  structure  and  processes  around  customer  demand  instead  of  around 
organizational  convenience. 

Womack  and  Jones  (2003)  discuss  the  need  to  overcome  organizational  inertia,  a 
force  associated  with  passive  resistance  to  change.  He  says  that  the  right  person  or 
persons  with  a  "make  something  happen"  mindset  are  necessary  to  act  as  “change 
agents,"  along  with  gathering  the  latest  in  knowledge  about  Lean  success  stories.  Lean 
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thinking  evidently  must  become  second  nature  to  everyone  in  the  organization  - 
culturally  ingrained  -  for  it  to  really  work. 

The  change  process  can  be  very  taxing  to  people  at  all  levels  of  the  organization. 
People  tend  to  resist  change  for  behavioral  and/or  systemic  reasons,  i.e.,  refusing  to 
change,  and  not  knowing  how  respectively.  Change  affects  people's  lives  and  introduces 
uncertainty.  Understanding  how  change  affects  people  in  the  organization  is  a  key 
component  of  managing  resistance  (Shere,  2006).  Thoroughly  communicating  the 
change  to  everyone  affected,  including  senior  leaders  modeling  the  desired  behaviors, 
capture  Kotter’s  (1996)  and  others  recommendations  for  successful  change. 

Creating  a  different  culture  is  transformative.  Unfortunately,  the  need  for  culture 
change  is  not  likely  to  be  driven  by  an  expressed  need  to  retain  knowledge  or  prevent 
knowledge  loss.  Recognizing  that  culture  change  should  be  addressed  is  more  complex, 
in  that  it  emerges  based  on  the  relative  fit  among  key  organizational  variables,  i.e., 
strategy,  structure  and  results.  According  to  DeLong  (1994),  knowledge-sharing 
behaviors  flourish  only  in  an  environment  where  there  is  a  sense  of  mutual  commitment 
between  the  organization  and  its  employees. 

6.  Command  and  Control  versus  System  Thinking 

The  previous  discussion  is  meant  to  show  that  there  is  more  to  transformation 
than  new  programs,  paying  a  contractor  for  good  ideas,  and  training.  The  challenge  for 
command-and-control  cultures  is  to  balance  the  need  for  control  with  new  needs  for 
speed,  innovation,  and  decentralized  decision-making.  Although  Denting  (1982) 
described  command  and  control  as  an  organizational  prison,  a  possible  alternative  is  to 
balance  a  strong  command  and  control  culture  with  guidance  gained  by  systems  thinking 
or  learning  organizations  (Senge,  1990). 

II.  ENTERPRISE  ASSESSMENT  METHODOLOGIES 

1.  Types  of  Assessment 

Assessment  is  defined  as  "the  evaluation,  or  estimation,  of  the  nature,  quality,  or 
ability  of  someone  or  something."  Evaluation  is  defined  as  "third  party  judgment  on 
the  worth  of  others'  performance,  project,  or  program"  (Humphries,  1998).  Evaluation  is 
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then  more  about  “success"  or  “failure"  according  to  criteria  external  to  the  organization 
being  assessed.  Being  judged,  especially  from  others  outside  the  trusted  workgroup,  can 
inhibit  learning  and  promote  “apparent  compliance"  rather  than  assessing  for 
organizational  learning  and  being  able  to  see  progress. 

Evaluation  appears  frequently  within  companies,  with  pressure  to  provide  proof 
or  value  to  the  business.  In  1998,  the  Society  for  Organizational  Learning  (SoL)  began 
an  initiative  to  study  assessment  and  how  it  either  builds  learning  within  the  organization, 
or  inhibits  it.  They  found  that  where  assessment  is  integrated  into  the  ongoing  procedures 
of  an  organization’s  activities,  it  becomes  a  tool  for  navigating  self-correction  or  making 
continuous  adjustments  (Humphries,  1998).  Three  basic  types  of  assessment  are 
described  below:  external  assessment,  internal  assessment,  and  self-assessment. 

a.  External  Assessment 

This  type  of  assessment  is  done  in  a  formal,  quantitative  format.  An 
example  of  an  external  enterprise  assessment  is  the  Malcolm  Baldrige  National  Quality 
Award  (Baldrige  National  Quality  Program,  2006).  Success  criteria  are  provided  to  an 
agency  applying  for  the  award,  and  external,  specially  trained  examiners  evaluate  the 
organization.  Through  a  structured  process  of  questions,  answers  and  observations, 
examiners  determine  how  the  organization  is  performing  against  the  criteria. 

Another  example  of  an  external  assessment  is  the  Capability  Maturity 
Model  Integration  (CMMI),  where  a  specially  trained  examiner  comes  to  the  company 
and  assesses  it  based  on  established  criteria,  usually  resulting  in  certification  levels, 
called  capability  or  maturity  levels  (Chrissis,  Konrad,  &  Shram,  2003). 

b.  Interna I  Assessment 

An  internal  assessment  can  be  the  same  tool  as  is  used  for  an  external 
assessment;  however,  in  this  case,  an  enterprise  would  manage  the  assessment  internally 
for  their  own  learning  or  recognition  of  value  of  the  achievements  in  process 
improvement.  The  enterprise  could  have  an  assessment  team  that  would  facilitate  the 
assessment  of  various  groups  or  teams. 
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c.  Self-assessment 

An  enterprise  can  choose  to  provide  a  tool  for  use  across  the  enterprise, 
defining  the  value  to  its  members,  and  those  members  within  a  team  or  group  would  use 
the  tool  to  self-assess  their  progress.  The  enterprise  can  then  obtain  the  results  from  all  of 
those  self-assessments,  or  just  let  the  entities  use  it  for  their  own  measure  of  progress. 

2.  Types  of  Quality  Programs 

Different  types  of  quality  programs,  certifications,  and  assessment  methodologies 
exist  in  industry,  government,  and  academia.  Awards  (such  as  Malcolm  Baldrige  Quality 
Award  and  the  Shingo  Prize  for  manufacturing),  inspections,  external  certifications,  and 
internal  self-assessments  exist  to  help  companies  ensure  that  their  strategics,  goals,  and 
objectives  are  tied  to  results. 

This  study  examines  four  assessment  mechanisms  in  particular:  MIT's  LAI; 
Malcolm  Baldrige  Quality  Program;  SEl's  CMMI;  and  the  International  Standards 
Organization  (ISO)  9000  quality  standard.  The  objective  is  to  evaluate  a  foundation  for 
which  self-assessment  tool  or  tools  could  be  used  by  NAE.  LAI  and  Baldrige  are 
examples  of  enterprise  tools,  while  CMMI  and  ISO  9000  are  normally  applied  within  an 
enterprise. 

3.  Lean  Aerospace  Initiative  (LAI) 

The  Lean  Aerospace  Initiative  (LAI)  started  in  1993,  and  is  a  consortium 
sponsored  and  managed  by  MIT.  The  purpose  of  the  LAI  consortium,  which  associates 
the  collective  needs  of  academia,  industry,  and  government,  is  to  provide  a  venue  for 
collaboration  with  the  objective  of  increasing  the  value  of  the  United  States  aviation 
industry.  As  a  consortium,  or  learning  network,  it  stimulates  new  forms  of  interaction 
(transformation)  by  associating  powerful  monuments  of  previous  organizational 
structures  such  as  customers,  suppliers,  workers,  management,  and  government 
(Murman,  et  al,  2002). 

The  consortium  members  come  from  airframe,  engine,  avionics,  missile  and  space 
companies,  military  service  program  offices.  Office  of  the  Secretaiy  of  Defense,  and 
MIT.  The  LAI  is  organized  with  member  organizations  as  its  base,  and  knowledge  teams 
and  networks  supporting  the  work  of  executive  committees  and  the  executive  board. 
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NAVAIR  currently  participates  on  the  executive  board.  The  member  organizations 
provide  the  executive  level  with  outcome  facts,  progress  and  issues,  and  research  needs. 


The  LAl's  work  began  in  the  I990's  by  pioneering  lean  thinking  and  practice. 
They  analyzed  the  elements  of  Lean  and  how  an  enterprise  is  transformed  by  applying 
lean  practices.  They  knew  that  leadership  was  critical  to  the  transformation,  and  defined  a 
leadership  model  of  setting  direction,  aligning  people,  and  motivating  and  inspiring  as 
key  factors.  LAI  brought  together  a  group  of  scholars,  industry  practitioners,  and 
government  officials  to  "operationalize"  those  elements  into  a  "roadmap"  framework  that 
would  translate  change  principles  into  guidelines  for  lean  enterprise  transformation. 
Their  “transition  to  lean"  roadmap  was  created  to  outline  the  steps  necessary  to  initiate, 
sustain,  and  refine  enterprise  transformation  (Murman,  et  al,  2002).  Figure  4  illustrates 
the  roadmap: 
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Figure  4.  LAI  Transition-lo-Lean  Roadmap  (From  Murman.  et  al.  2002) 


LAI  also  understood  that  an  enterprise  must  understand  its  current  and  desired 
future  state.  This  led  to  the  development  of  the  Lean  Enterprise  Self  Assessment  Tool 
(LESAT).  The  LESAT  has  since  been  adapted  for  use  by  government  organizations  as 
the  Government  LESAT  (GLESAT).  The  structure  of  the  lean  thinking  espoused  by  LAI 
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also  supports  other  systemic  change  initiatives,  such  as  total  quality  management.  Six 
Sigma,  and  process  reengineering,  as  is  seen  in  Figure  5. 
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Figure  5.  LAI’s  Lean  Thinking  Linkages  (From  Murman,  et  al,  2002) 

The  LAI  began  its  enterprise  level  work  in  1999  and  published  its  seminal  book 
on  the  subject  titled,  "Lean  Enterprise  Value:  Insights  from  MIT's  Lean  Aerospace 
Initiative"  (Murman,  et  al,  2002).  The  principles  driving  the  work  and  research  of  LAI  to 
create  lean  value  are: 

•  Doing  the  right  job  and  doing  it  right. 

•  Deliver  value  only  after  identifying  stakeholder  value  and  constructing 
robust  value  propositions. 

•  Lean  value  is  only  fully  realized  after  adopting  an  enterprise  perspective. 

•  Address  interdependencies  across  enterprise  levels  to  increase  lean  value. 

•  People,  not  just  processes,  effectuate  lean  value  (Murman.  et  al.  2002). 

The  mission  of  the  LAI  is  to: 

Research,  develop  and  promulgate  knowledge,  principles,  practices 
and  tools  to  enable  and  accelerate  the  envisioned  transformation  of  the 
greater  US  aerospace  enterprise  through  people  and  processes.  (Lean 
Aerospace  Initiative,  2005). 
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The  LAI  furthers  its  mission  through  support  of  industry  and  government 
transformat ion-to-lean  projects,  enabling  a  lean  value-creating  supplier  base,  education, 
training,  improving  effectiveness  of  enterprise  organizations,  and  sustainment  of  lean 
tools  and  knowledge  base  through  sponsored  outreach  events.  The  LAI  focuses  on  three 
knowledge  areas,  each  with  its  own  products  and  research  goals,  to  meet  current  and 
future  challenges: 

•  Enterprise  change  -  strategic,  enterprise-level  approach  to  accelerate 
effective  and  sustainable  improvement  and  change; 

•  Enterprise  architecting  -  create  a  future  lean  vision  and  designing  an 
enterprise  to  support  it;  incorporate  information,  organization,  process, 
technology  and  strategic  dimensions;  and 

•  Product  life-cycle  -  “Pushing  the  envelope"  in  the  area  of  designing  and 
development  aerospace  products  in  a  complex  system-of-systems 
environment  to  shorten  cycle  time,  reduce  cost,  and  increase  delivery  of 
best  lifecycle  value. 

The  value  of  the  LAI  can  be  seen  most  clearly  in  the  significant  gains  of  its 
members  and  prime  contractors  through  lean  implementations.  To  further  the  LAI  value 
proposition,  they  maintain  a  standard-setting,  forward-reaching,  research  program  that  is 
in  its  fifth  phase  of  development.  Phase  V  builds  off  of  the  previous  phase  IV  drivers, 
which  were: 

1 .  The  need  to  accelerate  lean  enterprise  transformation; 

2.  The  need  to  design  future  lean  enterprises;  and 

3.  The  need  to  evolve  adaptive  lean  transformation. 

As  such,  Phase  V  has  four  core  research  questions,  and  seven  research  threads, 
that  connect  to  knowledge  areas  in  Phase  IV.  The  core  research  questions  are: 

1.  How  can  1  understand  how  my  organization  /  enterprise  currently  operate 
within  its  larger  context? 

2.  How  can  I  define  and  evaluate  the  future  possibilities  for  evolving  a  more 
efficient  and  effective  enterprise? 
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3.  What  are  the  most  effective  strategies  and  tactics  to  achieve  these  future 
possibilities  for  my  enterprise? 

4.  I  low  can  1  best  manage  the  enterprise  change  process? 

The  seven  associated  research  threads  listed  below,  which  support  the  core 
questions,  are  necessary  to  achieving  the  LAI  vision  for  enterprise  transformation: 

1.  Actionable  characterization  of  lean  enterprise  attributes,  dynamics  and 
behaviors 

2.  Purposeful  architecting  of  enterprise  future  state 

3.  Value-driven  enterprise  decision  making 

4.  Effective  integration  of  enterprise  elements 

5.  Value-based  enterprise  performance  planning  and  assessment 

6.  Managing  uncertainty  in  complex  enterprises 

7.  Guiding  change  for  lean  enterprise  outcomes 

The  focus  of  this  thesis  is  to  define  the  best  methodology  for  an  enterprise 
assessment  of  AIR Speed  and  the  NAE,  with  additional  benefit  to  the  LAI  enterprise 
assessment  research  initiatives.  Specifically,  using  the  LAI  model,  the  thesis  attempts  to 
answer  knowledge  requirements  at  the  intersection  of  core  question  two  above  (How  can 
I  define  and  evaluate  the  future  possibilities  for  evolving  a  more  efficient  and  effective 
enterprise?),  and  research  thread  five  (Value-based  enterprise  assessment). 
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4.  Malcolm  Baldrige  Quality  Program 

The  Baldrige  National  Quality  Program  (BNQP)  is  named  for  Malcolm  Baldrige, 
who  was  a  former  Secretary  of  Commerce  under  President  Ronald  Reagan.  The  Baldrige 
Foundation  operates  both  the  BNQP  and  administers  the  Malcolm  Baldrige  Quality 
Award,  which  is  recognition  much  sought-after  by  companies  for  their  quality  processes 
and  products.  The  Baldrige  Foundation  created  the  Criteria  for  Performance  Excellence 
as  a  tool  to  help  businesses  understand  whether  or  not  they  were  making  progress  in 
deploying  their  strategy,  vision,  mission,  values,  and  plans.  The  Criteria  helps  companies 
answer  the  questions,  “are  we  making  progress,"  and  “how  do  we  know?” 

It  also  helps  organizations  learn  what  needs  improving.  While  other  approaches 
might  focus  on  a  single  aspect,  such  as  leadership,  strategic  planning,  or  process 
management,  the  Criteria  is  structured  into  an  integrated  management  framework  that 
addresses  factors  that  define  the  organization,  its  operations  and  its  results  (Baldrige 
National  Quality  Program  (BNQP),  2006). 

The  Criteria  consists  of  approximately  100  questions  grouped  into  an 
organizational  profile  and  seven  categories.  The  resulting  profile  is  a  snapshot  of  the 
organization  being  assessed,  and  compiles  the  key  influences  of  how  the  organization  is 
operated  and  the  challenges  it  faces.  Organizations  can  select  the  criteria  that  best  apply, 
such  as  a  business,  education,  health  care,  or  government.  The  business  criteria 
categories  are:  (BNQP,  2006) 

1 .  Leadership 

2.  Strategic  Planning 

3.  Customer  and  Market  Focus 

4.  Measurement,  Analysis,  and  Knowledge  Management 

5.  Human  Resource  Focus 

6.  Process  Management 

7.  Business  Results 

Figure  6  provides  the  framework  connecting  and  integrating  the  Categories. 
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Baldripe  Criteria  lor  Performance  Excellence  Framework:  A  Systems  Perspective 


Figure  6.  Framework  for  Baldrige  Criteria  for  Performance  Excellence  (  From  Baldrige 
National  Quality  Program.  2006) 

The  organizational  profile  sets  the  context  for  the  way  that  the  assessed 
organization  operates.  The  environment,  key  working  relationships,  and  strategic 
challenges  sen  e  as  a  guide  for  the  organization's  performance  management  system. 

The  six  Baldrige  Categories  in  the  center  of  Figure  6  above  cover  both  systems 
operations  and  results.  Leadership,  Strategic  Planning,  and  Customer  and  Market  Focus 
represent  the  leadership  triad,  which  are  placed  together  to  emphasize  the  importance  of  a 
leadership  focus  on  strategy  and  customers.  Senior  leaders  set  the  organizational 
direction  and  seek  future  opportunities  for  the  organization. 

Human  Resource  Focus,  Process  Management,  and  Results  present  the  results 
triad.  The  organization's  employees  and  key  processes  accomplish  the  work  of  the 
organization  that  yields  overall  performance  results. 
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All  actions  point  toward  Results,  which  is  a  composite  of  product  and  service, 
customer  and  market,  financial  and  internal  operational  performance  results,  including 
human  resource,  governance,  and  social  responsibility  results. 

The  leadership  triad  is  linked  to  the  results  triad  by  the  horizontal  arrow  in  the 
center  of  the  framework,  which  indicates  the  critical  central  relationship  between 
Leadership  and  Results  for  organizational  success.  The  two-headed  arrows  also  indicate 
the  importance  of  feedback  for  an  effective  performance  management  system. 

The  system  foundation  is  composed  of  Measurement,  Analysis,  and  Knowledge 
Management,  which  are  critical  to  effective  organizational  management  and  a  fact-based, 
knowledge  driven  system  for  improving  performance  and  effectiveness. 

The  2006  Criteria  include  19  questions,  one  for  each  of  the  Baldrige  Criteria  items 
(above).  Any  organization  can  use  the  Baldrige  Criteria  to  self-assess  its  performance  and 
improve.  The  Criteria  is  designed  to  help  align  resources,  improve  communication, 
productivity,  and  effectiveness,  and  achieve  strategic  goals.  If,  for  example,  the 
organization  is  using  tools  such  as  Lean  or  Six  Sigma,  the  Criteria  will  help  align 
resources  with  those  approaches.  The  Criteria  focus  on  common  requirements,  rather  than 
procedures,  tools,  or  techniques  in  the  following  key  areas  of  organizational  performance: 

1 .  Product  and  service  outcomes 

2.  Customer- focused  outcomes 

3.  Financial  and  market  outcomes 

4.  I  luman  resource  outcomes 

5.  Organizational  effectiveness  outcomes,  including  key  internal  operational 
performance  measures 

6.  Leadership  and  social  responsibility  outcomes 

These  measures  are  intended  to  ensure  that  an  organization's  strategies  are 
balanced  and  that  they  do  not  trade  off  among  important  stakeholders,  objectives,  or 
short-  and  longer-term  goals. 

The  Criteria  begins  with  the  Preface,  which  includes  an  Organizational  Profile, 
which  contains  both  a  description  of  the  organization  and  its  challenges.  The  assessment 
then  goes  into  each  Category'.  The  assessment  uses  a  point  system,  with  a  total  of  1000 
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points  possible.  Categories  are  broken  down  into  sub-categories.  For  example. 
Leadership  is  broken  down  into  Senior  Leadership  and  Governance  and  Social 
Responsibilities,  which  has  120  points  total.  Strategic  Planning  is  divided  into 
development  and  deployment. 

A  Self-Analysis  Worksheet  is  included  in  the  Assessment,  which  is  optional.  An 
organization  can  list  its  key  strengths  and  opportunities  for  improvement,  and  rank  them 
in  order  of  importance.  They  are  then  used  as  a  basis  for  creating  goals  and  a  plan  of 
action. 

The  scoring  system  uses  responses  to  Criteria  Items  and  is  based  on  two 
evaluation  dimensions:  process  and  results.  Criteria  users  furnish  information  relating  to 
these  two  dimensions.  "Process"  refers  to  the  methods  used  by  the  organization  to 
address  the  item  requirements  in  the  organizational  performance  areas  listed  above.  The 
four  factors  used  to  evaluate  process  are  approach,  deployment,  learning,  and  integration 
(ADLI). 

"Approach"  refers  to: 

•  The  methods  used  to  accomplish  the  process 

•  The  appropriateness  of  the  methods  to  the  item  requirements 

•  The  effectiveness  of  the  use  of  the  methods 

•  The  degree  to  which  the  approach  is  repeatable  and  systematic  (based  on 
reliable  data  and  information) 

"Deployment"  refers  to: 

•  The  approach  is  applied  in  addressing  item  requirements  relevant  and 
important  to  the  organization 

•  The  approach  is  consistently  applied 

•  All  appropriate  work  units  use  the  approach 

“Learning”  refers  to: 

•  Refining  the  approach  through  cycles  of  evaluation  and  improvement 

•  Encouraging  breakthrough  change  to  the  approach  through  innovation 

•  Sharing  refinements  and  innovations  with  other  relevant  work  units  and 
processes  in  the  organization 
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“Integration"  refers  to  the  extent  to  which: 

•  The  approach  is  aligned  with  organizational  needs  identified  in  other 
Criteria  Item  requirements 

•  Measures,  information,  and  improvement  systems  are  complementary 
across  processes  and  work  units 

•  Plans,  processes,  results,  analyses,  learning  and  actions  are  harmonized 
across  processes  and  work  units  to  support  organization-wide  goals 

“Results"  refers  to  the  organization’s  outputs  and  outcomes  in  achieving  the 
requirements  for  performance  and  improvement  in  the  key  areas  of  product  and  service 
outcomes,  customer  satisfaction,  financial  performance,  human  resource  outcomes, 
operational  performance,  and  leadership  and  social  responsibility.  Performance  levels  are 
compared  to  those  of  competitors  and  other  organizations  that  provide  similar  products 
and  services. 

The  four  factors  used  to  evaluate  results  are:  (a)  current  level  of  performance,  (b) 
rate  and  breadth  of  performance  improvements  (i.e.,  slope  of  trend  data  and  how  widely 
deployed  and  shared),  (c)  performance  relative  to  appropriate  benchmarks,  and  (d) 
linkage  of  results  measures  to  important  customer  performance  requirements  identified  in 
the  organizational  profile.  As  Process  items  mature,  their  description  should  indicate  the 
learning  that  occurs,  as  well  as  integration  with  other  processes  and  work  units.  Results 
items  call  for  data  showing  performance  levels,  improvement  rates,  and  relevant  data  for 
key  measures  and  indicators  of  organizational  performance.  There  should  be 
corresponding  results  if  improvement  processes  are  widely  shared  and  deployed.  Results 
scores  are  a  composite  based  on  overall  performance. 

Even  if  an  organization  docs  not  want  to  try  for  the  Baldrige  Award,  the  Criteria 
and  assessment  provide  a  way  to  assess  how  well  it  is  doing  and  a  guide  for  further 
improvement.  The  Criteria  apply  to  any  type  of  organization,  large  or  small,  service  or 
manufacturing,  and  help  to  measure  performance  and  planning  in  uncertain 
environments. 
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a.  U.S.  Coast  Guard  and  B.XQP 

The  U.S.  Coast  Ciuard  (USCG)  has  used  a  Baldrige-based  assessment 
process  since  1996.  They  credit  their  managerial  success  to  the  use  of  the  Criteria,  which 
gives  them  great  insight  into  their  management  practices.  The  thcn-Commandant 
established  an  internal  quality  award  program  and  a  framework  for  assessing 
performance  and  sharing  best  practices.  During  the  first  four  years,  the  award  process 
employed  a  written  application  and  examination  process  similar  to  Baldrige.  A  criteria 
guidebook  was  developed,  a  board  of  examiners  was  trained,  and  they  administered  the 
award  program. 

There  was  some  feeling  that  the  process  consumed  substantial  resources, 
allocating  one  half  of  one  year  to  the  application  process.  They  decided  to  simplify  the 
assessment,  with  an  emphasis  put  back  into  improvement,  de-emphasizing  the  award.  By 
the  end  of  2000,  Government  Executive  magazine  published  a  report  that  the  Coast  Guard 
was  the  only  federal  agency  to  receive  an  “A”  on  the  Federal  report  card  for  budget  and 
performance  management. 

In  2001,  USCG  unveiled  a  revamped  assessment  process,  emphasizing 
collaborative  learning  and  an  enterprise  focus.  They  still  had  qualified  examiners  in  both 
Baldrige  and  facilitation  behaviors.  The  entire  process  was  trimmed  to  three  days. 
Commanding  officers  liked  the  new  process  and  most  said  they  gained  valuable  insight 
into  their  own  unit  operations  and  made  improvements  to  their  managerial  processes 
based  on  the  assessment  results.  They  found  that  the  collaborative  process  provided  an 
opportunity  to  capture  valuable  learning  data,  as  examiners  recorded  systemic  issues  such 
as  organizational  barriers  or  challenges  that  impact  the  performance  of  best  work. 

The  most  recent  change  is  that  all  units  are  expected  to  assess,  plan,  and 
monitor  performance  using  the  Criteria.  The  assessment  process  was  deployed  to  every 
medium-to-large  unit  as  an  institutional  management  practice.  All  units  are  required  to 
participate  in  the  assessment  every  two  years.  There  are  still  challenges,  however. 
Resistance  to  the  assessment  process  still  exists,  and  scheduling  assessment  time  for  unit 
leaders  is  a  challenge. 
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The  Baldrige  process  has  helped  the  USCG  obtain  new  funding  from 
Congress  for  aging  ships  and  aircraft.  They  believe  that  the  assessment  process  is  a 
powerful  tool  to  educate  leaders  and  create  new  behaviors.  They  believe  it  can  be  used  in 
any  organization  to  promote  best  management  practices  (I it,  F.,  Kalnbach,  C.,  &  Smith. 
M.,  2003). 

5.  Capability  Maturity  Model  Integration  (CM MI) 

Capability  Maturity  Model  Integration  (CMMI)  is  a  process  improvement 
methodology  for  the  development  and  maintenance  of  products  and  services  throughout 
their  life  cycle.  When  organizations  are  seeking  an  appraisal  to  see  where  they  are  and 
how  they  can  improve.  CMMI  identifies  best  practices  that  an  organization  can  use  to 
address  the  development  and  maintenance  of  products  and  services  during  their  complete 
life  cycle. 

Industry,  government,  and  the  Software  Engineering  Institute  (SEI)  collaborated 
to  create  CMMI.  The  SEI  released  the  CMMI  Framework,  which  contains  a  full  set  of 
CMMI  models.  Based  on  the  improvement  needed,  an  organization  can  obtain  a  model 
for  systems  engineering,  software  engineering,  product  development,  etc.  One  has  to 
know  the  area  needing  improvement  and  the  content  of  the  model  to  use  them  effectively. 

SEI  recognizes  that  there  are  multiple  dimensions  to  a  business,  consisting  of 
people  (with  skills,  training,  and  motivation),  tools  and  equipment,  and  procedures  or 
processes.  Processes,  however,  are  what  hold  everything  together.  SEI  has  defined 
capability  maturity  models  (CMM)  that  focus  on  improving  process  management  in  an 
organization.  They  describe  an  evolutionary  improvement  path  from  ad  hoc,  immature 
processes,  to  disciplined  and  mature  processes  with  improved  quality  and  effectiveness. 

There  are  a  number  of  maturity  models,  standards,  and  methodologies  in  the 
marketplace,  however,  most  focus  on  a  specific  part  of  the  business.  CMMI  takes  a 
systemic  approach  to  the  problems  that  organizations  face.  CMMI  integrates  various 
"bodies  of  knowledge"  that  are  essential  when  developing  products,  such  as  systems 
engineering,  software  engineering,  and  acquisition.  This  approach  provides  a 
comprehensive  solution  to  an  organization. 
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CMMI  is  designed  for  product  life  cycle  improvement,  and  is  more  robust  than 
other  process  improvement  methodologies.  It  can  be  used  for  either  a  formal  certification 
or  simply  to  ensure  that  an  organization's  processes  are  disciplined  and  aligned,  with  a 
clear  path  toward  improvement. 

CMMs  have  been  developed  for  systems  engineering,  software  engineering, 
software  acquisition,  workforce  management  and  development,  and  integrated  product 
and  process  development  (Chrissis,  Konrad,  &  Shrum,  2003).  The  intent  of  CMMI  is  to 
provide  a  CMM  that  not  only  covers  product  and  service  development  and  maintenance, 
but  also  provides  an  extensible  framework  so  that  new  disciplines  can  be  added. 
Currently  four  disciplines  are  available: 

•  Systems  engineering  -  covers  the  development  of  total  systems,  with  or 
without  software 

•  Software  engineering  -  development  of  software  systems 

•  Integrated  product  and  process  development  (IPPD)  -  a  systematic 
approach  that  involves  all  stakeholders  to  provide  products  aligned  to 
customer  requirements 

•  Supplier  sourcing  -  covers  acquisition  of  products  from  suppliers 

Each  discipline  area  has  a  number  of  process  areas  within  it.  For  example,  within 
the  systems  engineering  discipline  there  are  22  process  areas,  e.g.,  configuration 
management,  measurement  and  analysis,  organizational  process  definition,  and  project 
planning. 

A  CMM  allows  for  different  approaches  for  the  models.  As  long  as  the  model 
contains  the  essential  elements  of  effective  processes,  and  describes  an  evolutionary 
improvement  path,  it  is  considered  a  CMM.  All  of  the  source  models  for  CMMI  are 
considered  CMMs,  yet  each  has  a  different  approach. 

Two  approaches  are  used  within  CMMI,  called  “representations”.  A 
representation  reflects  the  organization,  use,  and  presentation  of  components  in  a  model. 
All  CMMs  have  process  areas  that  are  defined  by  levels.  For  example,  for  the  process 
area  of  Project  Planning,  there  are  two  types  of  representations:  (a)  staged  and  (b) 
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continuous.  The  “staged"  representation  uses  predefined  sets  of  process  areas  to  describe 
an  improvement  path.  A  maturity  level  describes  this  improvement  path,  which  is  an 
evolutionary  plateau  toward  achieving  improved  processes.  The  “continuous" 
representation  is  used  with  the  systems  engineering  and  1PPD  disciplines,  and  allows  for 
selecting  a  specific  process  area  and  improvement  relative  to  it.  This  representation 
approach  uses  capability  levels  to  characterize  improvement  relative  to  an  individual 
process  area.  Table  1  shows  a  comparison  of  Capability  and  Maturity  levels. 

Table  1.  Comparison  of  Capability  and  Maturity  Levels 


Level 

Continuous  Representation 
Capability  Levels 

Staged  Representation 
Maturity  Levels 

0 

Incomplete 

N/A 

1 

Performed 

Initial 

2 

Managed 

Managed 

3 

Defined 

Defined 

4 

Quantitatively  Managed 

Quantitatively  Managed 

5 

Optimizing 

Optimizing 

As  is  shown  in  the  table,  there  is  a  different  starting  point  depending  on  which 
representation  is  chosen.  When  the  continuous  representation  is  chosen,  the  focus  is  on 
selecting  a  particular  process  area  and  the  granularity  of  improvement,  i.e.,  improving 
both  “what  we  do"  and  “how  we  do  it."  In  this  context,  whether  a  specific  goal  is 
performed  or  incomplete  is  important,  so  the  starting  point  here  is  “incomplete." 

Because  the  staged  representation  is  concerned  about  “maturity"  of  a  set  of 
processes,  whether  individual  processes  are  incomplete  or  performed  is  not  important. 
Therefore,  the  “initial"  name  is  given  to  the  starting  point  for  this  representation.  Table  2 
outlines  the  definitions  of  the  capability  levels. 
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Table  2. 


Definition  of  Capability  Levels 


Capability  Level 

Definition 

0  -  Incomplete 

A  process  that  is  either  not  performed  or  partially  performed 

1  -  Performed 

A  process  that  satisfies  the  specific  goals  of  the  process  area.  It 
supports  and  enables  the  work  needed  to  produce  work  products. 

2  -  Managed 

A  performed  (Level  I)  process  that  has  the  basic  infrastructure  in 
place  to  support  the  process.  It  is  planned  and  executed  in 
accordance  with  policy,  employs  skilled  people  who  have  adequate 
resources  to  produce  controlled  outputs,  involves  relevant 
stakeholders;  is  monitored,  controlled,  and  reviewed;  is  evaluated 
for  adherence  to  its  process  description 

3  -  Defined 

Managed  (Level  2)  process  tailored  from  standard  organizational 
processes,  and  contributes  work  products,  measures,  and  other 
process-improvement  information  to  the  organizational  process 
assets. 

4  -Quantitatively 
Managed 

Defined  (Level  3)  process  controlled  using  statistical  and  other 
quantitative  techniques.  Quantitative  objectives  for  quality  and 
process  performance  are  used  in  managing  the  process. 

5  -  Optimizing 

A  Level  4  process  that  is  improved  based  on  an  understanding  of 
the  common  causes  of  variation.  The  focus  is  on  continually 
improving  the  range  of  process  performance  through  both 
incremental  and  innovative  improvements 

Advancing  through  each  capability  level  is  accomplished  by  comparing  progress 
against  the  specific  criteria  in  each  level.  Each  level  should  be  reviewed  to  see  if 
progressing  is  applicable  and  of  relevance  and  value  to  the  organization. 

The  continuous  representation  approach  offers  a  more  flexible  approach  to 
process  improvement,  and  the  staged  representation  is  more  systematic,  offering  a  one- 
step-at-a-time  approach.  Both  representations  are  defined  to  offer  equivalent  results. 

One  of  the  elements  of  CMMI  is  process  institutionalization,  which  is  an 
important  concept  to  this  approach  to  process  improvement.  Institutionalization  implies 
that  a  process  is  ingrained  in  the  way  the  work  is  performed  and  there  is  commitment  and 
consistency  to  performing  the  process.  A  process  that  is  institutionalized  is  more  likely 
to  be  retained  under  stress. 
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CMMI  recognizes  that  process  improvement  efforts  must  be  tied  to  business  goals 
and  objectives.  Yet  CMMI  is  a  very  structured  and  rigorous  set  of  models,  designed  with 
a  goal  of  ensuring  that  process  areas  and  their  interrelationships  are  captured.  For 
example,  if  a  company  wanted  to  improve  the  time  to  produce  a  product,  they  might 
focus  on  the  project  management  process,  which  relies  on  best  practices  in  other  areas, 
such  as  project  planning  and  process  monitoring. 

a.  CMMI  Appraisal 

As  is  true  for  most  process  improvement  efforts,  there  are  principles  that 
are  applied  when  a  CMMI  appraisal  is  chosen.  These  principles  are: 

•  Senior  management  sponsorship 

•  Focus  on  business  objectives 

•  Confidentiality  for  interviewees 

•  Use  of  a  documented  appraisal  method 

•  Use  of  a  process  reference  model  (e.g.,  a  CMMI  model  as  a  base) 

•  Collaborative  team  approach 

•  Focus  on  actions  for  process  improvement 

The  most  rigorous  CMMI  appraisal  method  is  called  the  Standard  CMMI 
Appraisal  Method  for  Process  Improvement  (SCAMPI).  This  process  uses  external 
personnel  to  appraise  various  organizational  areas.  CMMI  also  allows  for  less  structured 
methods,  including  self-assessments  and  quick-look  or  mini-appraisals.  Links  to  a 
variety  of  useful  information  about  CMMI  are  available  on  the  SEI  website 
(http://www.sei.cmu.edu)  which  help  users  understand  the  various  methods  and  tradeoffs 
associated  with  each  method. 

6.  International  Standards  Organization  (ISO)  9000 

ISO  9000  series  standards  specifies  those  requirements  to  be  met  by  an 
organization's  quality  management  system  to  demonstrate  its  ability  to  provide  consistent 
products  that  meet  customer  requirements,  applicable  regulator)'  requirements,  and 
enhance  customer  satisfaction.  It  is  built  on  the  premise  that  every  organization  has 
processes,  and  what  makes  a  process  work  effectively,  can  be  identified,  controlled,  and 
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improved  (Russo  &  Russo,  1998).  The  ISO  9000  series  standards  help  organizations 
describe  the  processes  they  use  to  achieve  success.  Businesses  can  self-assess  using  ISO 
9000  or  they  can  apply  for  an  external  certification. 

As  with  many  quality  programs,  ISO  9000  has  its  roots  in  human  tragedy  and  a 
clearly  evident  need  for  improvement  known  as  “the  burning  platform."  Understanding 
this  history  is  important  to  an  understanding  of  the  ISO  process  and  methodology. 

During  World  War  II,  the  British  experienced  an  excessive  number  of  accidental 
detonations  in  their  weapons  factories  resulting  in  multiple  deaths  and  human  suffering 
that  could  not  longer  be  tolerated.  Responding  to  this  unacceptable  situation,  the  British 
government  dictated  the  use  of  a  standardized  production  process,  overseen  by 
inspectors,  that  was  to  be  implemented  by  any  vendor  hoping  to  supply  pails  for  British 
ordnance.  The  early  standards  were  seen  as  contractually  binding  obligations  (Russo  & 
Russo,  1998). 

In  similar  efforts,  the  United  States  developed  MIL-Q-9858A,  "Quality  Program 
Requirements,"  and  NATO  developed  "Allied  Quality  Assurance  Procedures"  to  improve 
the  consistency  of  military  hardware  bought  from  many  suppliers. 

By  the  late  1960's,  the  British  quality  movement,  with  the  support  of  the  British 
government,  moved  out  of  the  military  market  by  stating  publicly  that  “Quality  is 
everybody's  business."  In  1971,  the  British  Standards  Institute  published  the  BS  9000 
standard  for  the  electronics  industry,  and  in  1974,  they  implemented  a  more  general  BS 
5179,  "Guidelines  for  Quality  Assurance.”  The  BS  5 1 79  standard  moved  the  requirement 
for  inspection  from  the  customer  to  the  supplier  base  through  third-party  inspections. 
Through  several  revisions,  the  common  ISO  9000:1987  standard  was  derived  from  BS 
5750  with  a  focus  on  repeatable  factory  tloor  processes.  (British  Accreditation  Bureau, 
2006)  The  essential  purpose  of  BS  5750  was  a  demonstration  of  a  controlled  production 
process  for  quality  output,  sufficient  for  contractual  arrangements. 
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The  key  elements  were  (Russo  &  Russo,  1998): 

1.  Management  responsibility 

2.  Quality  systems 

3.  Contract  review 

4.  Design  control 

5.  Document  control 

6.  Purchasing 

7.  Customer  supplied  stock 

8.  Product  identification  and  traceability 

9.  Process  control 

10.  Inspection  and  test 

1 1 .  Inspection,  measuring,  and  test  equipment 

1 2.  Inspection  and  test  status 

13.  Control  of  non-conforming  product 

14.  Corrective  action 

15.  I landling,  storage,  packaging  and  deliver)' 

Satisfying  customers  and  reducing  waste  are  the  two  basic  tenets  of  the  ISO  9000 
program,  with  the  assumption  that  structured  quality  processes  and  assurance  tasks  will 
meet  the  expectations  of  the  program  tenets.  The  published  standards  promoted  control 
over  management,  as  well  as  providing  an  efficient  methodology  for  assessment  by 
inspectors.  In  simple  terms  they  demanded  that  firms,  “say  what  they  do,  and  then  do 
what  they  say”  (Russo  &  Russo,  1998).  The  British,  through  this  period,  saw  the 
“Japanese  Miracle”  (e.g.,  Toyota  Production  System).  They  tried  to  capture  the  essence 
of  the  Toyota  quality  programs,  but  failed,  because  they  did  not  see  the  unique  nature  of 
what  Toyota  did  —  which  was  to  think  differently  about  the  design  and  management  of 
work  (Seddon,  2002).  Toyota  did  try  an  ISO  approach  to  quality  in  one  of  its  factories, 
but  quickly  pulled  the  strategy  when  it  had  a  negative  influence  on  production. 

The  first  worldwide  quality  standard  was  attempted  in  1992  by  ISO  with  the 
publication  of  the  guidelines  for  the  ISO  9000  standard.  The  ISO  methodology  consists  of 
documenting  a  standard  approved  by  more  than  75  percent  of  the  ISO  member 
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organizations,  and  compliance  actions  by  organizations  that  desire  certification.  Finally,  a 
certified  registrar  audits  compliance  by  an  organization  to  the  standard  and  issues  a 
certification  with  periodic  reviews  to  maintain  compliance  standards.  The  ISO  9000 
series  of  standards  has  a  convoluted  naming  convention  with  numerous  recent  changes 
that  is  beyond  the  scope  of  this  thesis.  The  most  common  terminology  was  used  for  this 
study. 

The  ISO  9000  standard,  which  contains  ISO  9001 :2000,  has  five  sections: 

•  Quality  Management  System 

•  Management  Responsibility 

•  Resource  Management 

•  Product  Realization 

•  Measurement  Analysis  and  Improvement 

The  standard  requires  that  companies  write  an  organization-specific  ISO  9000 
quality  manual.  This  manual  defines  how  the  company  meets  each  requirement  of  the 
ISO  9000  standard,  with  further  definition  of  quality  procedures,  policy,  and  quality 
objectives.  The  executive  management  of  a  company  is  asked  to  define  the  quality  policy 
through  these  steps  (Russo  &  Russo,  1998): 

•  Define  the  quality  policy 

•  Establish  the  strategic  organizational  objectives 

•  Articulate  commitment  to  quality 

•  Ensure  that  everyone  in  the  organization  adheres  to  the  policy 

ISO  defines  a  quality  product  or  service  as  (Russo  &  Russo,  1998): 

•  The  result  of  an  organization's  customer  focused  processes 

•  Created  by  employees  that  understand  their  function  and  have  the  right 
resources 

•  Having  predictable  quality 
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The  certification  process  can  take  from  two  months  for  the  smallest  organization 
to  approximately  18-24  months  for  the  largest  organization.  The  process  can  be  grouped 
into  four  phases: 

1 .  Decision  to  get  ISO  certification  with  commitment  and  supporting  budget 

2.  Development  or  supporting  documentation  with  most  resources  being 
expended  in  this  phase.  Source  selection  and  contracting  for  an  ISO 
registrar  happens  at  the  end  of  this  stage. 

3.  Evolution.  This  is  where  the  workforce  adapts  to  the  documented 
procedures  developed  from  Phase  1  and  2. 

4.  Continuation.  An  ISO  registration  audit  is  conducted  and  non¬ 
conformance  issues  are  corrected  prior  to  certification.  Periodic  audits  are 
conducted  to  maintain  certification. 

The  challenges  for  enterprises  that  choose  to  use  ISO  9000  as  a  quality  program 
are  the  complexity  in  the  requirements  and  cost  for  certification.  Extensive  written 
documentation  can  often  be  required,  which  is  then  not  used  other  than  for  inspection 
purposes. 
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III. 


ASSESSMENT  TOOLS  ANALYSIS 


A.  INTRODUCTION 

This  thesis  analyzed  four  performance  assessment  tools  to  determine  their 
applicability  for  assessing  the  U.S.  Navy  AIRSpeed  initiative  under  the  Naval  Aviation 
Enterprise  (NAE).  The  research  methodology  included:  (1)  a  literature  review  of  four 
prominent  organizational  performance  assessment  tools,  and  others;  (2)  a  comparison  of 
the  four  tools  in  terms  of  organizational  fit  and  ease  of  implementation;  and  (3)  semi- 
structured  interviews  conducted  with  (10)  military  and  civilian  personnel  involved  with 
and/or  knowledgeable  of  Lean/Six  Sigma  principles.  Each  of  the  four  tools  is  described 
below. 

B.  DOD  CONTINUOUS  PROCESS  IMPROVEMENT  GUIDE 

On  15  May  2006,  the  Secretary  of  Defense  released  the  DoD  Continuous  Process 
Improvement  (CPI)  Transformation  Guidebook  (Deputy  Secretary  of  Defense,  2006)  as  a 
resource  for  designing  and  managing  CPI  efforts.  The  document  is  meant  to  codify 
experiences  using  CPI.  The  Guidebook  creates  a  common  lexicon  for  describing  and 
coordinating  individual  Service  CPI  efforts.  Its  design  incorporates  best  practices  from 
leading  industry  and  DoD  experience.  The  Deputy  Under  Secretary  of  Defense  (Logistics 
and  Materiel  Readiness)  (USD  (L&MR))  has  been  directed  to  take  action  to 
institutionalize  CPI  efforts  by  certifying  CPI  experts,  providing  guidance  documentation, 
and  enabling  collaboration. 

While  it  is  “labeled"  a  guide,  the  DoD  charges  the  USD  (L&MR)  to 
"institutionalize"  CPI.  The  definition  of  institutionalize  is  to  “make  something  an 
established  custom.”  It  is  anticipated  that  for  the  Services  to  achieve  institutionalization, 
more  than  an  invitation  to  follow  document  guidance  will  be  needed.  It  is  yet  to  be  seen 
what  specific  actions  will  be  taken  to  meet  this  direction.  (Deputy  Secretary  of  Defense, 
2006) 

The  DoD  CPITG  can  be  considered  a  broad-reaching  document  that  encompasses 
all  the  aspects  and  best  practices  of  CPI.  These  best  practices  include  long-term  cultural 
change  and  implementation  of  specific  tools  to  take  action  and  close  the  gap  from  an 
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organization's  current  state  to  a  desired  future  state.  The  document’s  overview  points 
largely  to  initial  “islands  of  success"  for  CPI  application  to  specific  projects  and 
organizations.  The  DoD  CPITG  goal  is  meant  to  connect  these  various  experiences  and 
to  instill  a  department-wide  culture  of  CPI  that  will  set  the  stage  for  greater  enterprise- 
level  improvements.  The  assessment  portion,  contained  in  Appendix  D.  is  discussed  later 
in  this  chapter  as  pail  of  the  appendix  review.  The  DoD  CPITG  has  four  areas  of  focus, 
listed  below  and  depicted  in  Figure  7: 

•  A  broad-based,  structured  CPI  implementation  method  to  focus  enterprise 
level  CPI  activities 

•  A  focus  on  CPI  implementation  within  a  structure  of  goals  that  are  aligned 
to  warfighter-driven,  outcome-based  performance  metrics  to  be  achieved 
at  the  lowest  total  cost 

•  Emphasis  on  the  management  and  integration  of  CPI  projects  with 
periodic  progress  evaluation  and  reporting 

•  Maturity  assessments  and  checklists  to  determine  how  well  projects  and 
organizations  are  progressing  with  CPI  initiatives,  training,  and 
certification. 

The  goal  of  the  journey  outlined  in  the  DOD  CPITG  is  to  create  an  “in  house” 
CPI  process  that  each  Service  owns  and  applies,  with  trained  experts  and  internal 
capability,  within  a  reasonable  time  frame,  e.g.  one  to  three  years. 


Figure  7.  Four  areas  of  DoD  CPITG  (From  Deputy  Secretary  of  Defense.  2006) 
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The  structure  of  the  document  contains  the  following  four  sections  (with  emphasis 
on  Sections  2  through  4).  depicted  in  Figure  8  below,  covering  the  introduction  of  CPI 
through  actual  assessment  and  analysis  tools  listed  in  the  appendices: 

•  Section  1:  Introduction  of  CPI  and  program  focus 

•  Section  2:  Framework  description  of  the  overall  concepts  as  well  as  a  DoD 
outline  for  CPI 

•  Section  3:  Specific  roles  and  responsibilities  of  participating  organizations 
and  elements  of  the  DoD  approach  to  CPI 

•  Section  4:  Attachments  covering  more  in-depth  material  on  specific  topics 
and  tools  that  would  be  useful  to  various  participants. 


Figure  8.  Pertinent  sections  of  the  DoD  CPITG 

(From  Deputy  Secretary  of  Defense,  2006) 

1.  DoD  CPITG  Framework  Pillars 

The  DoD  framework  has  evolved  from  three  schools  of  thought: 

1.  Lean  —  based  on  continually  working  to  minimizing  waste  while  coming 
closer  to  exactly  what  the  customer  desires1 

2.  Six  Sigma  --  evolved  from  statistical  analysis  and  quality  control  programs 

3.  Theory  of  Constraints  --  provides  an  advanced  planning  capability  through 
critical  chain  functionality. 

The  most  important  feature  of  the  DoD  CPITG  is  that  it  is  neither  prescriptive  nor 
directive  in  nature.  As  an  example,  it  speaks  to  the  importance  of  communication  on 


1  In  the  DoD  CPITG  the  customer  is  the  warfighter,  which  is  widely  interpreted  to  mean  single 
individuals  or  units  such  as  squadrons  for  the  NAE.  Sufficient  arguments  could  be  made  that  the  Combatant 
Commander  (COC'OM)  is  actually  the  customer  for  the  NAE  and  should  be  the  focus  for  CPI  initiatives. 
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information  management,  but  does  not  direct  the  specific  structure  of  these  processes  or 
information  systems  to  be  utilized.  The  DoD  CPITG  consists  of  five  "areas"  listed  below: 

a.  Area  I:  Fundamental  Concepts  of  CPI 

Several  areas  are  important  to  CPI  implementation  and  are  divided  into 
"musts,"  principles,  a  value  stream,  and  culture. 

CPI  "musts:" 

1.  Infrastructure  established  to  support  CPI  implementation  through 
designated  Champions,  Steering  Committees,  Support  Teams,  and 
Working  Groups 

2.  Goals  aligned  with  strategy,  mission  related,  and  adding  value  for 
customers 

3.  Apply  the  define,  measure,  analyze,  improve,  and  control  (DMAIC) 
process  with  a  CPI  plan  of  action  for  all  projects 

4.  A  visible  and  forceful  leadership  commitment  to  CPI  through  an 
innovative  and  collaborative  organization 

CPI  principles: 

1 .  Fact-based  data  analysis  to  determine  the  current  situation 

2.  Analysis  of  variation  from  known  standards 

3.  Utilize  systems  thinking  to  avoid  sub-optimization 

4.  Focus  on  value-added  elements  of  the  value  stream  such  as  people, 
machines,  and  systems 

5.  Controlled  continuous  experimentation 

6.  Based  on  long-term  improvement  as  the  overall  objective 

7.  Partner  with  suppliers,  customers,  and  internal  and  external  stakeholders 

The  successful  application  of  CPI  is  based  on  a  comprehensive  value 
stream  focus  for  the  organization  being  improved.  The  DoD  enterprise  is  extremely 
large,  complex,  and  globally  involved  with  thousands  of  personnel,  nodes,  interfaces, 
initiatives  and  activities.  For  CPI  to  be  effective,  the  value  stream  is  intended  to 
commence  at  the  enterprise  level,  to  manage  the  gaps  between  the  present  organization 
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and  a  desired  future  state.  An  example  of  a  transition-focused  enterprise  value  stream 
map  (Balazs,  James.  &  Parris.  2002)  is  shown  in  Figure  9. 


Figure  9.  Enterprise  Value  Stream  Mapping  Process  Example  (From  Deputy 

Secretary  of  Defense,  2006) 

b.  Area  2:  The  CPI  Deployment  Cycle 

The  DoD  CPITG  reflects  an  orderly  and  structured  process  for  the 
implementation  of  CPI  programs  using  a  four-phase  approach-  as  seen  in  Figure  10.  Each 
of  the  phases  is  designed  to  create  continuous  refinement  and  improvement  of  the 
previous  implementation  steps  to  build  a  foundation  for  overall  process  improvement.  As 
previously  shown,  the  DoD  CPITG  is  meant  to  keep  all  applicable  processes  strategically 
focused  with  the  appropriate  allocation  of  resources  to  meet  requirements.  The  result  is  a 
dynamic  document  and/or  roadmap  pointing  the  way  to  ongoing  improvements  adapted 
to  fit  changing  customer  needs  and  expectations. 

-  The  CPI's  four-phase  approach  is  modeled  after  the  MI  T  LAI  Transition-to-Lean  Roadmap  and 
directly  correlates  to  the  steps  within  that  framework. 
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RE-ENTRY 


IMPLEMENT 


1.  Develop 
Mission,  Vision 
&  Strategic  Plan 


Where  jm  I  going? 
Wtul  do  I  need  to 
get  done? 


2.  Conduct  a  Value 
Stream  Analysis 


3.  Develop 
Structure/Behavior 


4.  Goal  Alignment 
&  Deployment 

•  How  will  I  get 
people  to  do  it? 


Figure  10.  Deployment  cycle  for  CPI  programs  (From  Deputy  Secretary  of 
Defense,  2006) 


The  four  steps  in  the  deployment  cycle  are  defined  as: 

•  Develop  mission,  vision,  and  strategic  plan  —  answering  the  questions: 
Who  are  we?  Where  are  we  headed?  What  needs  to  get  done?  The 
strategic  vision  should  be  three  to  five  years  in  length. 

•  Conduct  a  Value  Stream  Analysis  —  to  see  the  beginning  of  value  creation 
through  to  customer  deliver)',  with  all  the  touch  points  in  the  process,  and 
process  stakeholders. 

•  Develop  structure  and  behavior  --  necessary  to  make  the  implementation 
happen  in  the  manner  expected.  Generally  this  is  enacted  through  a  llow- 
down  from  champion,  to  steering  committee,  to  support  team,  to  work 
group. 

•  Align  and  deploy  goals  —  know  what  will  motivate  employees  to  take 
correct,  timely  and  informed  actions  and  make  performance  assessment 
more  visible  and  measurable. 
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c.  Area  3:  Operational  Plan 

As  CPI  implementation  moves  out  of  the  planning  and  strategy  areas  of 
development  the  next  step  of  the  plan  is  to  define  actionable  tasks  that  are  achievable  and 
supported  by  all  the  senior  executive  leadership.  The  goals  are  to: 

•  Provide  actions  that  meet  the  transformation  objectives  for  the 
organization 

•  Capture  current  best  practices  as  a  foundation  to  minimize  the  amount  of 
change 

•  Provide  for  continuous  growth  in  the  CPI  implementation  to  capture 
expanding  areas  of  the  enterprise. 

d.  Area  4:  Change  Management 

The  DOD  CPITG  recognizes  the  fact  that  CPI  is  relatively  new  for  many 
defense  agencies,  and  will  likely  require  changes  across  the  workforce  to  meet  the 
demands  of  new  process  and  work  redesign.  To  adequately  address  the  needs  of  CPI,  a 
robust  change  management  structure  is  reinforced  in  the  literature  including  the  following 
actions: 

•  Educate  leaders  (continual  education  -  all  grades  and  ranks) 

•  Challenge  assumptions 

•  Secure  agreements 

•  Prepare  leaders,  staff,  and  organization  members  for  adopting  new  roles, 

applying  critical  thinking,  and  developing  and  measuring  streamlined 
processes 

•  Use  the  formal  process  of  DMAIC  to  define,  measure,  analyze, 
implement,  and  control. 

To  facilitate  a  large-scale  change  process,  many  organizations  will  need  to 
transition  from  their  "as  is"  to  their  desired  "to  be"  state,  including  being  able  to  measure 
and  show  visible  changes  to  the  workforce.  Making  improvements  visible  is  meant  to  act 
as  a  motivating  and  momentum-generating  force.  The  outcomes  and  improved  designs  of 
one  cycle  provide  the  stalling  point  for  the  next  cycle  of  improvement. 
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c.  Area  5:  Metrics 

There  is  probably  no  more  important  attribute  of  a  successful  CPI 
implementation  than  measuring  progress.  The  metrics  constitute  the  lens  or  the  criteria 
through  which  success  will  be  measured.  The  structure  and  encouragement  of  a  metrics 
system  demonstrates  the  priorities  of  senior  leadership.  The  process  of  collecting  data  for 
the  metrics  process,  and  translating  data  into  usable  knowledge  so  leaders  and  managers 
can  enact  changes,  describes  a  non-ending  cycle  of  process  improvement. 

The  criteria  for  evaluating  metrics  should  be: 

•  Valid  to  measure  the  intended  process  attribute. 

•  Obtainable  in  a  timely  manner  with  allocated  resources. 

•  Accurate  and  repeatable,  earning  the  trust  of  all  involved. 

•  Relevant,  timely,  and  correlated  to  the  actions  of  the  organization. 

•  Consistent,  relevant,  periodic  cost  measurement  using  logical  and 
transparent  cost  constructs. 

Other  considerations: 

•  Level  of  aggregation  boundaries  matter,  i.e.,  defined  such  that  there  is 
comparative  consistency  across  the  organization.  Cost  metrics  are 
sensitive  and  can  be  difficult  to  define. 

•  Data  ownership  and  sharing  roles  matter,  i.e.,  clear  roles  defined  for  who 
has  responsibility  and  authority  for  data  collection,  maintenance  and 
security,  including  information-sharing  protocols. 

2.  Roles  and  Responsibilities 

•  Champion  -  a  designated  relatively  senior  person  assigned  to  perform 
change  agent  activities,  sponsored  by  and  having  access  to  senior 
executive  leadership,  and  knowledgeable  of  organizational  history, 
mission,  vision,  goals,  and  CPI  principles. 

•  Steering  committee  -  a  senior,  cross-functional  team  responsible  for 
aligning  the  new  direction  of  CPI  with  organizational  design  elements, 
including  identifying,  measuring  and  obtaining  results. 
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•  Support  teams  -  managers  who  provide  training  and  facilitate  DMAIC 
project  management  of  CPI  initiatives. 

•  Work  groups  —  process  owners  who  employ  appropriate  CPI  tools  to  the 
given  situation. 

•  Peer  groups  —  share  common  functional  responsibilities  and  provide  an 
opportunity  for  sharing  information,  challenges,  approaches,  activities, 
and  accomplishments. 

Within  organizations,  these  roles  and  responsibilities  can  be  resident  at  different 
levels.  Senior  leaders  set  the  new'  CPI  direction,  ensure  appropriate  resources  and  identify 
desired  results,  managers  modify  and  improve  work  design  around  various  functions,  and 
all  employees  learn  and  apply  the  CPI  methodology. 

Other  important  items  to  consider: 

•  Information  technology  personnel  and  vendors  manage  crucial  data  in  a 
different  way,  i.e.,  result-  and  knowledge-oriented,  secure,  and  not 
complicated  by  “rice  bowl"  behaviors. 

•  Contracting  is  carefully  integrated  so  that  organizational  cultural  changes 
revolve  around  critical  core  processes. 

•  Facilities  management,  planning,  and  budgeting  personnel  should  have 
early  and  consistent  involvement  because  waste  reduction  in  operations 
can  create  offsetting  waste  increases  in  reconfiguring  facilities. 

•  Human  resource  management  increases  in  importance  in  terms  of  the 
criticality  of  recruiting,  hiring,  training  and  promoting  the  right  people. 
CPI  cannot  /  must  not  be  perceived  in  terms  of  downsizing  and/or  jobs 
reduction,  or  the  process  will  suffer  the  consequences  of  cultural 
alienation. 

•  Supply  chain  management  increases  in  importance  to  ensure  a  linking 
relationship  among  process  owners. 

•  Labor  union  personnel  should  have  early  and  consistent  involvement  to 
obtain  insights  and  to  avoid  excessive  confrontation. 
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•  Finance  and  budget  management  teams  should  ensure  that  monetary 
resources  are  timed  to  be  available  when  needed  to  support  CPI  initiatives. 

•  Engineering  skills  increase  in  importance  at  various  levels  of  the  CPI 
effort  and  should  be  connected  to  hiring  and  training  initiatives. 

3.  DoD  CPITC  Attachments 

The  DoD  CPITG  attachments  below  provide  procedures,  tools,  and  checklists  to 
support  CPI  efforts.  Attachment  D,  as  the  assessment  piece  of  the  DoD  CPITG,  is  an 
essential  attachment  for  the  purposes  of  this  thesis  and  is  described  at  the  end  of  this 
section. 

1.  Resources:  provides  lists  of  applicable  books  and  organizations  that  can 
help  to  build  explicit  and  tacit  knowledge  necessary  to  understand  and 
successfully  apply  CPI  activities. 

2.  Organizational  Implementation  Planning  Framework:  provides  a  checklist 
for  CPI  steps  and  actions  as  seen  in  Table  3. 

3.  Training  and  Certification:  provides  steps  to  facilitate  the  understanding 
and  skills  in  the  core  competencies  necessary  for  successful  launch  and 
sustainment. 

4.  Community  of  Practice  Progress  Assessment:  provides  techniques  for 
assessing  CPI  performance  and  maturity  level. 

5.  CPI  Toolbox:  provides  evolution  of  CPI  techniques  and  common  tools. 

6.  Sample  CPI  Project  Selection  Criteria  and  Project  Charter:  provides  tools 
to  implement  specific  CPI  actions. 

7.  Terminology:  provides  recommended  common  lexicon  critical  for 
information  sharing  across  the  enterprise. 
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Table  3. 


List  of  CPI  steps  and  associated  actions  (From  Deputy  Secretary  of 

Defense,  2006) 


CPI  Step  Action  description 

Develop  Enterprise  Strategic  Plan  1 .1  Get  Leadership  Commitment 

1.2  Obtain  Expert  Help 

1.3  Identify/Select  Steering  Committee  and  Support  Team 

1.4  Identify  Strategic  Goals,  Vision.  Metrics 

1.5  Convey  Urgency/Burning  Platform 

1.6  Commit  to  Develop  People 

1.7  Focus  on  Customer  Value 

1.8  Communicate 

1.9  Develop  a  Transformation  Strategy 

1.10  Identify  and  Assign  Deployment  Actions 

1.11  Conduct  a  Risk  Assessment 

1.12  Identify  Funding/Resource  Requirements  and  Sources 

Conduct  a  Value  Stream  Analysis  2.1  Establish  an  Enterprise  Approach 

2.2  Perform  Customer/Stakeholder  Analysis 

2.3  Value  Stream  Map  Key  Processes 

2.4  Identify  Leverage  Points 

2.5  Select  Pilot  Projects 

2.6  Determine  How  to  Measure  Benefits 

Develop  Structure  and  Behavior  3.1  Conduct  Initial  Training 

3.2  Identify  and  Select  Additional  Champions 

3.3  Establish  CPI  Support  Infrastructure 

3.4  Use  the  Steering  Committee  to  Share  Lessons 

3.5  Organize  for  CPI  Implementation 

Align  and  Deploy  Goals  4.1  Deploy  Aligned  Goals  Down  Through  Organization 

4.2  Align  Sub-Organizations  Commitments  to  Goals 

Develop  an  Operational  Plan  5.1  Align  Plan  to  Strategy 

5.2  Establish  Deployment  Funding/Resources 

5.3  Establish  Timing  and  Priorities 

5.4  Identify  the  Improvement  Activities 

5.4  Address  Retention/Improvement  of  Output  Quality 
Implement  the  Operational  Plan  6.1  Conduct  Initial  Improvement  Activities 

6.2  Baseline  Key  Processes 

6.3  Focus  on  Standardization:  Policies.  Procedures, 
Processes 

6.4  Focus  on  Quality 

Monitor  Progress  7.1  Monitor  Performance  to  Established  Organization  Goals 

7.2  Coach  for  Results 

Focus  on  CPI  8.1  Sustain  Gains 

8.2  Nurture  the  Process 

8.3  Establish  a  Cross-Feed  Process 

8.4  Ensure  Senior  Leader  Participation 

8.5  Conduct  a  Self-Assessment 

8.6  Update  Direction  and  the  Plan 
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4.  DoD  CPITG  Attachment  D  -  CPI  Progress  Assessment  Tool 

The  DoD  CPITG  includes  three  forms  of  assessment: 

1 .  A  quick  ten-question  assessment 

2.  A  maturity  assessment  tool  for  overall  evaluation  of  organizational  CPI 

3.  An  advanced  assessment  tool  that  is  intended  for  use  by  organizations 
striving  for  world-class  excellence 

The  quick  assessment  form  is  simply  a  list  of  questions  with  no  scoring.  The 
overall  and  advanced  assessment  tools  are  based  on  a  maturity  model  with  scoring  on  a 
one-to-five  scale.  The  advanced  assessment  includes  score  weighting  to  express  the 
leadership's  relative  value  in  different  CPI  functions.  Additionally  the  advanced  tool 
provides  a  process  for  gap  analysis  to  a  future  preferred  state.  Each  tool  is  designed  to  be 
administered  internally,  or  facilitated  by  an  external  third  party,  to  yield  a  measure  of  the 
organizations  current  state  and  CPI  achievement. 

The  “quick"  ten  question  assessment  covers  the  following  topics  and  can  be  very 
effective  in  early  CPI  efforts,  with  few  resource  demands,  to  determine  the  as-is  state  of 
CPI. 

1.  Immediate  impressions  of  “5s”3  in  one  implementation  for  every 
organizational  area 

2.  Performance  against  transparent  and  aligned  CPI  metrics  recognizable  by 
randomly  selected  employees. 

3.  Visible  kaizen  (acceleration  of  process  improvement)  events  being 
conducted  with  quantification  of  the  results. 

4.  Random  assessment  of  the  value  added  work,  to  non-value  added  work, 
ratio  in  any  selected  core  processes. 

5.  Observe  ratio  of  work  in  progress  to  visible  inventory. 

6.  Are  actions  of  the  CPI  champion  visible? 

7.  Does  cross-functional  teaming  predominate  over  “stovepipe"  activities? 

8.  Are  value  stream  maps  visible  and  continually  refined? 

9.  Is  CPI  education  and  training  in  place  to  the  benefit  of  the  employees? 

10.  Are  the  islands  of  CPI  success  connected  across  the  enterprise  or 
functioning  in  isolation? 

1  Based  on  Japanese  words  that  begin  with  "S,"  the  5S  philosophy  focuses  on  effective  work  place 
organization  and  standardized  work  procedures.  The  five  S's  are:  sort,  set  in  order,  shine,  standardize,  and 
sustain.  (George,  2003) 
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The  overall  CPI  maturity  assessment  is  more  in-depth  but  still  very  functional  as 
an  internal  assessment  that  can  give  faster  results.  The  overall  assessment  has  a  one-to- 
five  scoring  that  rate  an  organization  as: 

1.  Not  Yet  Started 

2.  In  Process 

3.  Demonstrated 

4.  High  Visibility 

5.  Transformation  Evident 

The  scores  for  the  overall  assessment  are  assigned  across  the  following  criteria 
and  sub-criteria: 

Planning 

1.  Mission,  Vision,  and  Strategic  Planning 

2.  Conduct  a  Value  Stream  Analysis 

3.  Develop  Structure  /  Behavior 

4.  Goal  Alignment  and  Deployment 

Implementation 

1 .  Create  and  Refine  Operational  Plan 

2.  Implement  Operational  Plan 

3.  Monitor 

4.  Focus  on  Continuous  Process  Improvement 

The  resultant  cumulative,  average  score,  from  each  criterion  is  ranked  as  follows: 

•  Less  than  three  =  beginner  stage  of  CPI 

•  Three  to  Four  =  intermediate  stage  of  CPI 

•  Four  to  Five  =  advanced  stage  of  CPI 

The  advanced  CPI  assessment  tool  goes  into  greater  detail  to  support  the  weighted 
scores,  and  gap  analysis  necessary  to  manage  the  actions  of  an  organization  striving  for 
world  class  excellence.  It  is  based  on  a  tool  used  extensively  for  similar  purposes  by  John 
Allen  at  Total  Systems  Development  (TSD)  and  includes  concepts  from  LAI  and  the 
Shingo  Prize.  It  is  a  leadership  guide  as  well  as  progress  measurement  process. 

In  this  model,  leadership  is  asked  to  describe  their  desired  organizational  state  in 
two  to  three  years.  The  resultant  scores  become  the  benchmark  against  which  the  current 
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state  is  measured.  The  assessment  must  be  led  from  the  top  down  from  individuals  who 
have  the  vision  and  influence  to  sustain  the  early  change  effort.  The  structure  of  the 
advanced  assessment  tool  ranks  22  areas  of  CPI,  as  seen  in  Table  4,  on  a  scale  of  one  to 
five  for  current  state,  future  state,  gap  between  the  two  states,  and  evaluator  rating.  The 
final  scores  are  displayed  on  an  eight-axis  spider  graph  (shown  in  Figure  1 1)  to  visually 
display  the  current  state  of  the  enterprise. 


Table  4.  Twenty-Two  CPI  Areas 


Para.  No. 

CPI  Area 

1.1 

Vision 

1.2 

Commitment 

1.3 

Policy  Deployment 

2.1 

Leadership 

2.2 

Steering  Committee 

2.3 

Support  Team 

3.1 

Team-Based  Workgroups 

3.2 

Aligned  Organization 

3.3.1 

PDCA  Management  Process 

3.3.2 

Empowering,  Facilitating,  Coaching  Styles 

3.4.1 

Training 

Compensation  and  Recognition 

mm 

Selection  Process 

4.1.1 

Employee  Attitudes  towards  Improvement 

4.1.2 

Cooperation,  Trust  and  Respect 

4.1.3 

Employment  Security 

4.2.1 

Leaming/Continuous  Improvement 

Suggestion  Systems 

4.3.1 

Selection  Process 

Certification 

5.1 

Workplace  Organization 

5.2 

5S  Discipline  for  a  Clean  Workplace 
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Figure  1 1 .  Eight  axis  spider  graph  (From  Deputy  Secretary  of  Defense,  2006) 


C.  NAVY  PERFORMANCE  EXCELLENCE  GUIDEBOOK  (NPEG) 

On  May  3,  2006,  the  Secretary  of  the  Navy,  Donald  C.  Winter,  issued  a 
memorandum,  "Transformation  through  Lean  Six  Sigma"  (Winter,  2006)  In  that 
memorandum,  Mr.  Winter  outlined  the  Navy's  issues  of  fighting  a  war  and  positioning 
the  naval  forces  for  the  future,  combined  with  fiscal  pressures  urging  wise  use  of 
taxpayer  dollars.  Previously  in  industry,  he  knew  of  examples  where  buyers  and  sellers 
who  employed  Lean  Six  Sigma  (LSS)  experienced  better  efficiencies  and  higher  levels  of 
performance.  lie  sent  a  clear  message  that  he  expects  leadership  to  support  the  LSS 
initiative  by  "injecting  it  into  performance  objectives.”  His  goal  is  to  fully  employ  LSS 
in  Navy  organizations. 

To  assist  Navy  organizations  with  their  transformation  to  lean,  the  Office  of  Fleet 
Readiness  and  Logistics  (N4),  Office  of  the  Chief  of  Naval  Operations  (OPNAV),  has 
drafted  the  Navy  Performance  Excellence  Guidebook  (NPEG).  The  NPEG  as  of  this 
writing  is  in  final  form,  awaiting  CNO  signature.  The  NPEG  provides  Navy  leaders  with 
an  "easy-to-use"  framework  for  improving  performance  excellence,  intended  to  guide 
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them  through  their  continuous  process  improvement.  The  NPEG  is  considered  to  be  a 
“strategic  imperative"  for  implementing  Sea  Enterprise.  It  will  help  answer  the 
questions,  “Do  we  know  what  we  are  doing?  Do  we  need  to  be  doing  it?  Does  it  add 
value?  How  do  we  measure  success?"  (Navy  Performance  Excellence  Guidebook,  2006) 

The  NPEG  is  a  non-mandatory  tool  for  Navy  organizations  that  complements 
existing  enterprise  maturity  efforts.  Goals  of  the  NPEG  are  to: 

•  Help  leaders  self-asscss  their  management  system  performance 

•  Coalesce  the  results  into  a  Strategic  Plan  (Execution  Plan  included) 

•  Identify  opportunities  for  improvement,  and  leverage  core  strengths 

•  Jump-start  and  integrate  change  initiatives 

•  Provide  consistency  of  focus  over  time 

•  Institutionalize  a  common  framework  for  improvement. 

Intended  outcomes  of  using  the  NPEG  are  to:  reduce  non-value  added  activity, 
address  areas  that  need  improvement,  and  identify  and  leverage  core  strengths.  It  states  it 
will  help  organizations  to: 

•  Manage  knowledge  better 

•  Boost  organizational  learning 

•  Integrate  and  energize  improvement  initiatives 

•  Create  value  within  organizations. 

A  crucial  variable  to  the  success  of  applying  the  NPEG  is  senior  leadership 
understanding  and  involvement.  Senior  leaders  are  recommended  to  take  ownership  of 
the  outputs  (Strategic  Plan  and  Self-Assessment). 

The  Navy  used  validated  management  practices  to  create  the  Performance 
Excellence  Process  and  Self-Assessment  Guide.  Some  of  those  practices  were  drawn 
from  the  Malcolm  Baldrige  Quality  Award  criteria  and  capability  maturity  matrix 
models. 
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The  NPEG  defines  five  performance  excellence  process  steps,  which  are  shown  in 
Figure  12.  The  document  is  organized  into  five  sections,  corresponding  to  the  five  steps: 
Step  1 :  Organizational  Profile:  Who  we  are? 

Step  2:  Organizational  Assessment:  What  we  do. 

Step  3:  Strategic  Plan:  Where  we  are  going? 

Step  4:  Execution  Plan:  How  we  get  there. 

Step  5:  Performance  Measurement:  How  we  measure  our  progress. 

Each  section  describes  the  step,  and  provides  a  process  table  that  defines  the  steps 
to  completion,  outputs,  and  considerations. 
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Figure  12.  Navy  Performance  Excellence  Process  (From  U.S.  Navy,  2006) 

There  are  two  major  outputs  of  the  process:  self-assessment  results  and  a 
strategic  plan.  Even  if  some  organizations  are  already  doing  strategic  planning  or  some 
self-assessment,  the  purpose  of  the  NPEG  is  to  help  integrate  those  activities  into  a 
integrated  and  aligned  management  system  that  does  not  perpetuate  waste. 
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There  are  seven  appendices  in  the  NPEG: 


Appendix  A  - 
Appendix  B  - 

Appendix  C  - 
Appendix  D  - 

Appendix  E  - 

Appendix  F  - 
Appendix  G  - 


Glossary  of  terms 

Outlines  the  performance  criteria  and  the  points  available  for  each 
area  of  the  criteria.  This  might  be  a  more  readable  way  to  see  how 
each  criteria  area  is  weighted. 

Criteria  response  guidelines,  including  the  ADLI  evaluation 
factors,  taken  from  the  Baldrige  criteria 

Plan  of  Action  and  Milestones  Worksheet.  Identifies  key 
strengths,  opportunities  for  improvement,  and  non-value  added 
activities.  Used  after  the  self-assessment. 

Examples  of  outputs  and  additional  clarification  for  what  is 
required  to  document  for  each  step 
Describes  how  the  self-assessment  was  developed 
Resources;  readings,  websites,  and  online  courses 


1.  Timelines  for  using  the  NPEG 

The  NPEG  provides  a  suggested  process  timeline  (Figure  13)  and  how  much  time 
is  expected  for  each  step: 


W*  J!k  1  Weeks  Z-4  Win**  31?  Wi  inks  13  02 


Rep  •  at 


Figure  13.  Suggested  Timeline  for  NPEG  Steps  (From  U.S.  Navy,  2006) 
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2.  Step  1  -  Organizational  Profile 

The  starting  point  for  the  process  is  the  organizational  profile.  The  profile 
outlines  the  organization,  its  relationships,  operating  environment,  and  the  key  challenges 
it  faces.  It  helps  delineate  what  is  important  to  the  organization,  key  influences,  and 
where  the  organization  is  headed.  It  provides  a  context  for  understanding  the  organization 
before  undertaking  a  self-assessment. 

The  suggested  methodology  for  creating  this  profile  is  to  assemble  a  small, 
credible  group  within  the  organization  or  enterprise  to  answer  defined  questions.  The 
questions  are  divided  into  two  major  sections:  organizational  challenges  and 
organizational  description. 

The  organizational  challenges  section  addresses  the  environment  in  which  the 
organization  operates  and  its  important  strategic  challenges.  It  also  addresses  how- 
organizational  learning  and  performance  improvement  is  approached.  There  are  a  total  of 
six  questions  in  this  area. 

The  organizational  description  area  identifies  key  characteristics  and  relationships 
that  shape  the  organizational  environment.  They  also  address  governance,  and  there  are 
nine  questions  in  this  section. 

The  profile  enables  an  objective  depiction  of  the  organization  and  its  context,  i.e., 
why  it  exists,  and  where  it  wants  to  go  in  the  future.  It  is  designed  to  assist  leaders  and 
managers  in  making  strategic  decisions  that  will  shape  the  organizations  future.  The 
process  steps  for  creating  the  profile  are  provided  in  the  document,  as  well  as  the  primary- 
outputs  and  considerations.  (Chief  of  Naval  Operations  (CNO),  2006) 

3.  Step  2  -  Organizational  Assessment 

The  organizational  assessment  analyzes  the  organization's  strengths,  weaknesses, 
opportunities,  and  capabilities.  It  indicates  whether  the  organization  is  achieving 
performance  targets.  It  also  identifies  capabilities  that  the  organization  must  develop  to 
become  more  effective  and  efficient.  There  are  two  necessary  initial  steps,  which 
typically  take  two  to  three  w'eeks  prior  to  actually  conducting  a  self-assessment.  First, 
senior  leadership  identifies  whether  the  whole  or  part  of  the  organization  will  participate 
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in  the  self-assessment,  and  a  small,  cross-functional  team  is  formed.  Then,  the  team  gets 
familiar  with  the  NPEG  and  evaluates  the  organization  using  the  scoring  tables  and 
spreadsheets  that  accompany  the  document. 

The  process  to  be  used  for  assessing  an  organization  is  then  outlined  in  the 
document.  The  self-assessment  is  divided  into  five  process  steps,  which  are  the  same  as 
in  the  Performance  Excellence  Process.  A  scoring  table  is  provided  for  each  question. 
They  create  a  composite  score  for  each  criterion.  Scores  are  then  compiled  to  determine 
maturity  levels  for  all  five  steps  of  the  Process.  There  are  a  total  of  95  sets  of  self- 
assessment  questions. 

The  self-assessment  guide  is  built  on  a  set  of  core  values  and  concepts: 

•  Visionary  leadership  -  directions  are  set  and  there  is  a  focus  on  the 
customer;  leaders  are  role  models  for  their  people. 

•  Customer-driven  excellence  -  the  customer  assesses  quality  and 
performance,  so  attention  to  customer  requirements  demand  constant 
sensitivity. 

•  Organizational  and  personal  learning  -  learning  is  part  of  the  culture; 
knowledge  is  shared;  directed  to  better  outputs  and  services. 

•  Valuing  personnel  and  partners  -  committing  to  their  well-being  and 
satisfaction;  encourage  risk-taking  and  innovation;  build  partnerships. 

•  Agility  -  the  capacity  for  rapid  change  and  flexibility. 

•  Focus  on  the  future  -  understand  the  factors  that  affect  the  organization. 

•  Managing  for  innovation  -  becomes  part  of  the  learning  culture. 

•  Management  by  fact  -  collect  and  analyze  the  data  that  supports  good 
decisions. 

•  Social  responsibility  -  stress  responsibilities  to  the  public,  ethical 
behavior. 

•  Focus  on  results  and  creating  value  -  create  balance  and  value  to 
stakeholders. 
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•  Systems  perspective  -  manage  the  congruence  of  variables  over  which 
management  has  full  or  partial  control  to  allow  and  encourage  cultural 
adaptation  to  new  ways  of  achieving  and  measuring  success. 

It  is  noteworthy  that  most  of  these  core  values  and  concepts  match  those  found 
and  cited  in  the  Chapter  2  literature  review. 

The  self-assessment  is  divided  into  the  same  five  steps  as  the  Performance 
Excellence  Process,  and  four  criteria  are  examined: 

1.  Senior  Leadership  -  key  aspects  of  responsibilities,  and  how  they 
communicate  and  set  the  vision  and  values; 

2.  Governance  and  social  responsibility  -  outlines  key  aspects  of  the 
organization's  governance  system  and  how  it  fulfills  public 
responsibilities; 

3.  Customer  and  Mission  Area  Knowledge  -  key  processes  for  gaining 
knowledge  about  current  and  future  customers,  mission  areas,  outputs  and 
services,  and  how  pace  is  kept  with  changes;  and 

4.  Customer  Relationships  and  Satisfaction  -  what  processes  exist  for 
building  customer  relationships  and  determining  satisfaction? 

The  criteria  and  key  questions  for  the  steps  1-5  of  the  Self-Assessment  are 
outlined  in  Table  5.  Some  amplification  for  each  criteria  and  question  is  provided,  as 
well  as  the  definitions  for  five  levels  to  be  used  for  scoring  criteria.  The  criteria  start  at  a 
low  level  and  advance  towards  “perfection.”  For  example,  the  Question  A  Level  1  and 
Level  5  scoring  criteria  for  Criteria  1.1,  Senior  Leadership,  Vision  and  Values  is: 

“How  do  your  leaders  approach  their  responsibility  for  setting  expectations  and 
deploying  the  mission?" 

•  Level  1  —  We  have  no  clear  organizational  mission,  vision,  or  values. 
Leadership  focuses  on  the  organization's  short-term  survival,  and  manages 
by  directive.  Senior  leadership  initiatives  are  driven  by  personal  career 
motivations.  Employees  are  expected  to  follow  shifting  direction  and 
handle  daily  problems  as  they  arrive.  Suppliers  and  partners  are  not 
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engaged  in  any  way  with  the  organization's  mission.  "Kissing  up  and 
kicking  down"  is  the  norm. 

•  Level  5  —  All  employees  have  a  strong  identification  with  the  mission, 
vision,  and  values  of  the  organization  and  are  committed  to  its  goals. 
Senior  leadership  is  consistent  in  initiatives  and  direction  regardless  of 
changes  in  leadership.  People  think  and  act  like  “owners  of  the  business," 
in  their  decision-making  and  communications.  An  infectious  continuous 
improvement  culture  has  evolved  within  the  organization.  Employees  are 
confident  in  their  ability  to  pursue  learning,  make  decisions,  and  make 
changes  to  improve  their  organization,  confident  that  their  decisions  will 
be  supported." 
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Table  5.  Self-Assessment  Criteria  and  Questions 


Step 

Criteria 

Sub-Criteria 

Question 

1.1 

Senior  Leadership 

Vision  and  Values 

•  How  do  your  leaders  approach  their  responsibility  for  setting 
expectations  and  deploying  the  mission? 

•  I  low  do  senior  leaders  promote  an  environment  that  fosters  and 
requires  legal  and  ethical  behavior? 

•  How  do  the  organization’s  leaders  create  a  sustainable  and  relevant 
organization  with  performance  improvement,  accomplishment  of 
strategic  objectives,  innovation,  and  organizational  agility? 

•  How  do  leaders  personally  participate  in  succession  planning  and  the 
development  of  future  organizational  leaders? 

Communication  & 

Organizational 

Performance 

•  How  do  the  organization's  leaders  promote  improvement, 
communicate  with  personnel,  promote  teamwork,  and  empower 
others  to  do  what  needs  to  be  done? 

•  How  do  the  organization's  leaders  focus  on  action  to  accomplish 
objectives,  improve  performance,  and  attain  their  vision? 

1.2 

Governance  &  Social 
Responsibility 

Organizational 

Governance 

•  I  low  does  the  organization  evaluate  its  governance  system? 

•  How  does  the  organization  evaluate  performance  of  senior  leaders 
and  how  do  the  leaders  use  performance  evaluations  to  improve  their 
leadership? 

Legal  &  Ethical 
Behavior 

•  How  does  the  organization  anticipate,  evaluate,  and  rectify  the 
adverse  impacts  of  community  operations  and  the  environment? 

•  How  does  the  organization  use  mechanisms  for  promoting  and 
monitoring  of  ethical  behavior  by  all  personnel?  How  are  breaches 
handled?  ^ 

Support  of  Key 

Local  Communities 

•  How  does  the  organization,  particularly  senior  leaders,  actively 
support  and  help  to  improve  local  communities? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

1.3 

Customer  &  Mission 

Area  Knowledge 

Customer  & 

Operational 

Knowledge 

•  1  low  well  does  the  organization  know  its  customers  and  relate 
capabilities  to  these  customers? 

•  How  well  does  the  organization  use  feedback  from  its  customers 
(internal,  external)  regarding  value,  cost,  and  quality  to  become 
more  customer  focused? 

•  How  well  does  the  organization  keep  listening  and  learning  methods 
current  with  operational  needs  and  directions,  including  changes  in 
external  environment? 

1.4 

Customer  Relationships 
&  Satisfaction 

Customer 

Relationship 

Building 

•  How  do  you  build  relationships  with  customers  to  meet  and  exceed 
their  expectations,  while  improving  financial  performance? 

•  I  low  does  the  organization  respond  to  customer  comments  and 
complaints,  ensure  that  it  resolves  them  effectively,  and  perform 
improvement  efforts? 

Customer 

Satisfaction 

Determination 

•  How  does  the  organization  track  customer  satisfaction  and  how  does 
it  use  customer  problems  to  improve  processes? 

•  1  low  do  you  determine  your  organization’s  customer  satisfaction 
levels? 

•  1  low  do  you  keep  your  approaches  to  determining  satisfaction 
current  with  operational  needs  and  directions? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

2.1 

Measurement,  Analysis 
&  Review  of 
Organizational 
Performance 

Performance 

Measurement 

•  How  does  the  organization  select,  collect,  align,  and  integrate  its 
metrics  to  track  daily  performance  and  achievement  of  strategic 
objectives  and  action  plans?  To  what  extent  do  managers  and 
employees  understand  and  use  performance  metrics  to  monitor 
processes? 

•  I  low  do  you  select  and  ensure  the  effective  use  of  key  comparative 
data  for  operational  and  strategic  decision-making  and  innovation? 

1  low  do  you  measure  cost  savings? 

•  How  docs  the  organization  keep  its  performance  measurement 
system  current  with  its  operational  needs  and  directions? 

Performance 
Analysis  & 

Review 

•  I  low  does  the  organization  review  and  analyze  organizational 
performance  and  capabilities?  How  does  it  assess  its  ability  to 
respond  to  changing  organizational  needs  and  challenges  in  the 
operating  environment? 

•  I  low  does  the  organization  use  organizational  performance  reviews 
as  inputs  to  the  strategic  planning  process  and  innovation? 

2.2 

Information  & 

Knowledge  Management 

Data  & 

Information 

Availability 

•  How  do  you  make  needed  data  and  information  available?  How  do 
you  make  them  accessible  to  personnel,  suppliers,  partners, 
customers,  and  stakeholders? 

•  How  does  the  organization  develop  and  ensure  reliable,  secure,  and 
user-friendly  hardware  and  software  availability? 

•  How  does  the  organization  keep  its  data  and  availability 
mechanisms  current? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Criteria 

Sub-Criteria 

Question 

Organizational 

Knowledge 

Management 

•  How  does  the  organization  maximize  performance  through 
knowledge  management?  How  does  the  organization  use 
management  tools  and  methods  for  transferring  workforce 
knowledge? 

Data,  Information, 

&  Knowledge 
Quality 

•  How  does  the  organization  ensure  that  performance  data, 

information,  and  organizational  knowledge  are  accurate,  accessible, 
reliable,  timely,  and  secure? 

Strategic  Development 

Strategy 

Development 

Process 

•  1  low  does  the  organization  use  a  systematic  strategic  planning 
process  to  guide  decisions  and  continuously  improve  performance? 

•  I  low  does  the  organization  identify  strengths,  weaknesses, 
opportunities,  and  threats  during  its  strategic  planning  process? 

Strategic 

Objectives 

•  How  does  the  organization  define  strategic  objectives  and  establish 
timetables  for  their  accomplishment? 

•  1  low  do  the  organization’s  strategic  objectives  specifically  address 
the  organizational  challenges  in  its  Organizational  Profile? 

•  How  does  the  organization’s  strategic  planning  process  incorporate 
planning  for  change  management?  How  does  it  address  the  cultural 
challenges  the  organization  may  face  in  executing  the  strategic  plan? 

Work  Systems 

Organization  & 
Management  of 
Work 

•  1  low  does  the  organization  reduce  waste  and  costs,  improve 
customer  satisfaction,  embrace  innovation,  promote  cooperation, 
and  reward  initiative? 

•  I  low  does  the  organization  develop  and  implement  systems  that 
enable  it  to  capture  ideas  from  diverse  personnel  and  knowledge 
from  external  sources? 

•  How  does  the  organization  encourage  open  and  effective 
communication  and  skill  sharing  across  all  work  units? 
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Table  5.  Self- Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

Personnel 

Performance 

Management 

System 

•  1  low  does  your  personnel  performance  management  system, 
including  feedback  to  personnel,  support  high-performance  work 
and  contribute  to  the  achievement  of  your  Execution  Plans? 

•  1  low  does  your  personnel  performance  management  system  support 
a  customer  and  readiness  focus?  How  do  your  compensation, 
recognition,  and  reward  and  incentive  practices  reinforce  high- 
performance  work? 

Hiring  and  Career 
Progression 

•  How  do  you  identify  characteristics  and  the  skills  needed  by 
personnel? 

•  How  do  you  recruit,  hire,  and  retain  new  personnel?  How  do  you 
ensure  that  the  personnel  represent  the  diverse  ideas,  cultures,  and 
thinking  needed  for  your  desired  workforce? 

•  How  do  you  accomplish  effective  succession  planning  for  leadership 
and  management  positions?  I  low  do  you  manage  effective  career 
progression  for  all  employees  throughout  the  organization? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

4.2 

Personnel  Learning  & 
Motivation 

Personnel 

Education 

Training  and 
Development 

•  1  low  does  the  organization  use  education  and  training  of  personnel 
to  achieve  objectives  and  address  key  needs  associated  with 
organizational  performance  management? 

•  How  does  the  organization  address  key  organizational  needs  in  such 
areas  as  new  personnel  orientation,  diversity,  ethical  practices, 
workplace  environment  and  safety,  and  management  and  leadership 
development? 

•  How  does  the  organization  receive  input  on  its  personnel's 
education,  training,  and  development  needs? 

•  1  low  does  the  organization  deliver  education  and  training  programs 
to  personnel? 

•  How  does  the  organization  reinforce  on  the  job  knowledge  and  skills 
of  personnel  to  ensure  that  they  retain  knowledge  and  skills  for  long¬ 
term  organizational  use? 

•  How  do  you  evaluate  the  effectiveness  of  education  and  training, 
taking  into  account  individual  and  organizational  performance? 

Motivation  and 
Career 

Development 

•  I  low  do  you  involve  personnel  to  motivate*  them  and  to  develop 
and  utilize  their  full  potential? 

•  How  does  the  organization  help  its  personnel  attain  their  job,  career, 
and  learning  objectives? 

4.3 

Personnel  Well-Being 
and  Satisfaction 

Work 

Environment 

•  1  low  do  you  ensure  workplace  preparedness  for  disasters  or 
emergencies? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 

Sub-Criteria  Question 

Personnel  Support  •  How  does  the  organization  monitor  the  well-being,  satisfaction,  and 
and  Satisfaction  motivation  of  its  personnel? 

•  How  does  the  organization  tailor  its  personnel  services,  benefits,  and 
policies  to  the  needs  of  a  diverse  workforce? 

•  1  low  does  the  organization  develop  measures  and  indicators  of 
personnel  well-being,  satisfaction,  and  motivation? 

•  1  low  do  you  relate  assessment  findings  to  key  results  to  identify 
_ priorities  for  improving  the  work  environment  and  personnel  support? 

•  I  low  does  the  organization  identify,  understand,  and  define  success 
for  all  of  its  key  outputs,  services,  and  value  creation  processes? 

•  1  low  does  the  organization  identify  and  meet  key  requirements  for 
value  creation  processes  and  continuous  improvement  techniques? 

•  How  does  the  organization  incorporate  new  technology, 
organizational  knowledge,  and  agility  into  its  value  creation? 

•  How  does  the  organization  use  productivity,  efficiency  and 
effectiveness  measures  in  the  design  and  implementation  of  its  value 
creation  processes? 

•  With  respect  to  value  creation  processes,  how  do  you  foster 
innovation  and  continuous  improvement  (Cl)  at  all  levels? 

•  I  low  does  in-process  management  ensure  key  process  outputs  are 
met?  How  is  stakeholder  input  used  for  in-process  management? 

•  How  does  the  organization  minimize  the  cost  of  quality? 

•  How  does  the  management  system  handle  and  dispose  of  non- 
conforming  process  output? 

•  I  low  does  the  organization  capture  and  reinvest  cost  savings? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step  Criteria 


.5  Support  Processes 


Sub-Criteria  Question 


•  How  does  the  organization  determine  its  key  support  processes? 

•  How  does  the  organization  use  a  system  to  ensure  that  processes  meet 
all  key  requirements  and  that  new  technology  and  organizational 
knowledge  are  incorporated  into  such  processes? 

•  How  does  the  organization  utilize  a  wide  range  of  efficiency  and 
effectiveness  measures  in  the  design  of  its  support  processes?  How 
does  it  implement  processes  to  ensure  that  the  effects  are  desirable  and 
that  they  meet  design  requirements? 

•  How  does  your  day-to-day  operation  of  key  support  processes  ensure 
meeting  key  performance  requirements? 

•  How  does  the  management  system  incorporate  input  and  ensure  that 
in-process  measures  of  support  processes  are  monitored? 

•  How  does  the  internal  auditing  system  for  support  processes  work  in 
your  organization?  In  addition,  how  does  the  organization  use 
preventive  activ  ities  to  minimize  overall  cost  of  quality? 

•  How  does  the  quality  management  system  handle  non-conforming 
support  process  output  and  adjustment  to  any  corrective  and 
preventative  action  processes  (if  any)? 

•  With  respect  to  support  processes,  how  do  you  foster  innovation  and 
continuous  improvement  (Cl)  at  all  levels  in  the  organization? 

•  To  what  extent  does  the  organization  have  an  appropriate  system  to 
provide  adequate  financial  resources  and  alignment  to  objectives? 

•  How  does  the  organization  assess  and  plan  for  risks,  including 
financial  risks  and  emergencies? 

•  How  does  management  recognize  what  portion  of  its  cost  is  direct 
labor,  material,  overhead,  general  and  administrative? 


Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

4.6 

Strategy  Deployment 

Execution  Plan 
Development  and 
Deployment 

•  How  does  the  organization  develop  Execution  Plans  with  customers 
and  stakeholders,  and  evaluate  whether  those  plans  meet  their 
operational  expectations? 

•  1  low  does  the  organization  base  its  human  resources  plans  upon 
short-  and  long-term  implementation  objectives  and  plans? 

•  1  low  does  the  organization  use  key  performance  measures  for 
tracking  the  progress  of  Execution  Plans,  reinforcing  our 
organizational  alignment,  and  addressing  all  key  mission  areas  and 
stakeholders? 

Performance 

Projection 

•  1  low  does  the  organization  track  short-  and  long-term  performance 
projections  using  the  same  metrics  that  we  use  for  tracking  the 
progress  of  Execution  Plans? 

5.1 

Output  &  Service 

Results 

•  1  low  does  the  organization  track  current  levels  and  trends  in  key 
measures  of  output  and  service  performance  that  are  important  to 
customers? 

5.2 

Customer-Focused 

Results 

•  I  low  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  customer  satisfaction  and  dissatisfaction? 

•  How  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  customer-perceived  value? 

5.3 

Financial  Results 

•  1  low  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  financial  performance?  In  addition,  how 
does  the  organization  identify  cost  drivers  and  link  costs  to  outputs? 

•  How  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  operational  performance? 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

5.4 

1  luman  Resource  Results 

•  How  does  the  organization  track  work  system  performance  and 
effectiveness  of  its  human  resources  with  measures/indicators? 

•  1  low  does  the  organization  track  indicators  of  personnel  learning 
and  development? 

•  1  low  does  the  organization  track  current  levels  and  trends  in  key 
measures  of  personnel  well  being,  satisfaction,  and  dissatisfaction? 

5.5 

Organizational 
Effectiveness  Results 

•  1  low  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  the  operational  performance  of  your  key 
value  creation  processes?  This  includes  productivity,  cycle  time, 
supplier  and  partner  performance,  and  other  appropriate  measures  of 
effectiveness  and  efficiency 

•  How  does  the  organization  track  current  levels  and  trends  in  key 
measures  or  indicators  of  the  operational  performance  of  your  other 
key  processes  (such  as  administrative,  support,  or  financial 
operations)?  This  includes  productivity,  cycle  time,  supplier  and 
partner  performance,  and  other  appropriate  measures  of 
effectiveness  and  efficiency 
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Table  5.  Self-Assessment  Criteria  and  Questions  (cont.) 


Step 

Criteria 

Sub-Criteria 

Question 

5.6 

Leadership  &  Social 
Responsibility  Results 

•  1  low  does  your  organization  make  use  of  metrics  for  tracking  the 
effectiveness  of  your  organizational,  change  management,  and 
Execution  Plan  strategies? 

•  1  low  does  the  organization  measure  and  track  indicators  of  ethical 
behavior  and  stakeholder  trust  in  the  senior  leaders  and  overall 
governance  and  also  how  does  it  track  indicators  of  breaches  ethical 
behavior  by  all  levels  of  employees? 

•  1  low  does  the  organization  track  trends  in  key  measures  or 
indicators  for  internal  and  external  fiscal  accountability? 

•  1  low  does  the  organization  look  at  measures  or  key  indicators  of 
regulatory  and  legal  compliance?  In  addition,  how  does  the 
organization  look  for  important  trends  in  compliance? 
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4.  Step  3  --  Strategic  Plan 

The  NPEG  provides  an  organization  with  clear  guidance  showing  how  to  produce 
a  Strategic  Plan  as  an  output.  It  is  anticipated  that  the  final  Plan  will  propose  a  three-  to 
five-year  path  toward  the  organization's  vision.  The  process  for  developing  such  a  Plan 
is  provided,  as  well  as  the  primary  outputs  and  considerations. 

The  NPEG  does  not  provide  explicit  “instructions"  for  completing  the  strategic 
plan,  but  does  provide  an  outline  or  template  for  how  it  could  be  completed.  Benefits  of 
strategic  planning  are  also  described.  Suggestions  for  when  planning  should  be  done, 
preparing  for  planning,  using  a  facilitator  to  help  with  the  planning,  roles  in  the  process, 
planning  process  examples,  and  how  to  ensure  execution  of  the  plan  are  given.  They 
suggest  that  the  strategic  plan  be  updated  every  three  years,  visiting  it  annually  through 
an  “execution  plan”. 

5.  Step  4  —  Execution  Plan 

One  of  the  outputs  from  the  strategic  planning  activity  is  an  Execution  Plan, 
which  is  the  current  year's  focus  defined  in  specific  actions  for  the  organization.  It 
provides  functional-level  detail  that  supports  the  organization's  vision,  strategy,  and 
goals.  The  process  for  developing  an  execution  plan  is  provided,  as  well  as  the  primary 
outputs  and  considerations.  A  template  is  provided  that  outlines  the  suggested  contents 
of  an  effective  execution  plan.  It  becomes  “section  2”  of  the  Strategic  Plan. 

The  Execution  Plan  contains  a  description  of  where  an  organization  is  headed  in 
the  current  year  (“where  we  are  headed"),  as  they  relate  to  goals  in  the  strategic  plan. 
There  is  an  operations  section  (“how  we  do  it"),  which  provides  an  understanding  of  how 
the  organization's  services  or  outputs  are  created  and  delivered  to  customers.  A  financial 
plan  (“how  we  pay  for  it")  is  included  that  indicates  a  cost  profile  and  metrics  to  monitor 
financial  performance.  The  plan  management  section  ("how  we  update  it")  serves  as  a 
framework  for  execution  and  fostering  continuous  improvement.  it  defines  the 
accountability  needed  to  keep  the  plan  active. 
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6.  Step  5  --  Measure  Performance 

This  section  describes  a  short  process  for  how  performance  of  goals  and 
objectives  is  measured.  It  provides  the  steps  to  completion,  primary  outputs,  and 
considerations  used  for  this  process,  as  well  as  for  the  other  steps.  There  are  suggested 
metrics  provided  in  the  Appendix  E. 

The  NPEG  is  a  very  thorough,  comprehensive  approach  to  defining  how  to  assess 
an  organization,  from  the  creation  of  the  organizational  profile,  development  of  metrics, 
and  the  self-assessment  methodology. 

D.  GOVERNMENT  LEAN  ENTERPRISE  SELF-ASSESSMENT  TOOL 

(GLESAT) 

In  2001,  the  LAI  developed  the  Lean  Enterprise  Self-Assessment  Tool  (LESAT) 
(Murman,  et  al,  2002),  which  employs  a  capability  maturity  model  to  measure  the 
"leanness"  of  an  organization.  Their  focus  was  to  have  a  tool  that  would  measure  the 
enterprise,  versus  a  particular  process.  LAI  felt  that  traditional  measures  were  inadequate 
in  portraying  progress  toward  achieving  behavior  changes  or  improvement  strategics.  The 
team  at  MIT  compared  the  categories  of  various  assessment  tools  such  as  the  Malcolm 
Baldrige  National  Quality  Award  and  ISO  9004.  They  also  held  discussions  with 
Carnegie-Mellon  regarding  the  CMMI.  They  agreed  on  the  value  of  the  capability 
maturity  model  for  defining  progressively  greater  levels  of  capability,  as  an  organization 
"matures”  in  its  performance  of  a  particular  factor. 

The  LESAT  development  team  gathered  extensive  requirements  from  LAI 
member  organizations  regarding  their  needs  relative  to  assessment.  From  there,  LAI 
developed  common  a  set  of  user  requirements:  the  assessment  tool  should  be  flexible  for 
organizational  scope,  be  easy  to  understand  and  apply,  must  assess  degree  of  "leanness” 
for  an  enterprise  and  its  core  processes,  and  must  provide  feedback  for  improvement  and 
next  steps. 

A  number  of  aerospace  organizations  participated  in  the  testing  of  LESAT. 
Various  degrees  of  success  were  achieved,  but  they  all  realized  that  the  lean  journey  was 
one  of  years,  not  months.  As  government  agencies  began  participating  in  the  assessment. 
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LAI  realized  that  another  tool  was  needed,  tailored  specifically  to  the  government 
environment.  The  LESAT  was  modified  and  the  GLESAT  became  available  in  2005. 

1.  GLESAT  Structure 

The  GLESAT  is  based  on  the  enterprise  process  architecture  and  the  LAI 
Transition-to-Lean  Roadmap  described  in  Chapter  2.  The  GLESAT  is  organized  into 
three  assessment  sections: 

•  Lean  Transformation/Leadership  -  the  process  and  leadership  attributes 
nurturing  the  transformation  to  lean  principles  and  practices; 

•  Lifecycle  Processes  -  the  processes  responsible  for  the  product  from 
conception  through  post-delivery  support;  and 

•  Enabling  Infrastructure  -  the  processes  that  provide  and  manage  the 
resources  enabling  enterprise  operations. 

There  is  a  rating  system  of  one  to  five,  with  level  one  being  some  awareness  of 
lean  practices  and  some  ongoing  improvement  efforts.  Level  five  shows  the  enterprise 
doing  exceptionally  well  in  applying  lean  principles  and  practices,  and  being  innovative 
in  deploying  it  in  the  enterprise.  The  example  below  (Murman,  et  al,  2002)  shows  five 
capability  levels  for  the  factor  “Lean  Enterprise  Vision": 

Level  1  -  Senior  leaders  have  varying  visions  of  Lean,  from  none  to  well  defined 

Level  2  -  Senior  leaders  adopt  common  vision  of  Lean 

Level  3  -  Lean  vision  has  been  communicated  and  is  understood  by  most 

employees 

Level  4  -  Common  vision  of  Lean  shared  by  the  extended  enterprise 

Level  5  -  All  shareholders  have  internalized  the  Lean  vision  and  are  an  active  part 

of  achieving  it 

The  GLESAT  maturity  matrices  are  organized  as  follows,  with  a  total  of  54  lean 
practices  to  be  assessed  according  to  the  five-level  scale: 
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Section  1  -  Lean  Transformation/Leadership 
1  A.  Enterprise  Strategic  Planning  (3  lean  practices) 

IB.  Adopt  Lean  Paradigm  (4  lean  practices) 

IC.  Focus  on  the  Value  Stream  (4  lean  practices) 

I D.  Develop  lean  structure  and  behavior  (7  lean  practices) 

IE.  Create  and  refine  transformation  plan  (3  lean  practices) 

IF.  Implement  lean  initiatives  (2  lean  practices) 

IG.  Focus  on  continuous  improvement  (5  lean  practices) 

Section  2  -  Life-Cycle  Processes 

2A.  Set-up  the  enterprise  (4  lean  practices) 

2B.  Build  relationships  (3  lean  practices) 

2C.  Develop  the  plan  (5  lean  practices) 

2D.  Implement  the  plan  (4  lean  practices) 

2E.  Learn,  improve,  and  sustain  (2  lean  practices) 

Section  3  -  Enabling  Infrastructure 

3A.  Lean  organizational  enablers  (5  lean  practices) 

3B.  Lean  process  enablers  (3  lean  practices) 

2.  Assessment  Methodology 

To  ensure  effective  assessment,  LAI  recommends  a  structured  methodology  to 
administer  the  GLESAT,  including  semi-annual  or  annual  assessment.  There  are  five 
recommended  steps: 

1.  A  facilitated  meeting  to  introduce  the  tool,  and  provide  materials  to  the 
team.  Enterprise  leaders  should  be  at  this  meeting. 

2.  Enterprise  leaders  examine  GLESAT  and  conduct  assessment. 

3.  Leadership  reconvenes  to  discuss  results  and  determine  the  organization’s 
present  maturity  level. 

4.  Leadership  determines  their  desired  maturity  level  and  measures  the  gap. 

5.  An  action  plan  is  developed  and  resources  defined  and  prioritized. 

Figure  14  shows  the  process  definition,  criteria  structure,  lean  practices,  and 

characteristics  of  the  GLESAT,  created  and  maintained  by  MIT  LAI. 
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Government  LESAT  Enterprise  Self-Assessment  Tool  (Government  LESAT)  1 .0 


SECTION  I  SUMMARY  5HEEI  -  LEAN  IKANSFORMATION/LEADERSHIP 

Process  Definition:  Develop  and  deploy  loan  implomonfation  plans  throughout  iho  enterprise  loading  to  (1)  long-torm  sustainability,  (2) 
acquiring  competitive  odvantago,  and  (3)  satisfaction  of  stakeholders. 


TTL  Link 

Lean  Practice  Lean  Characteristic 

Capability  level 

Current 

Desired 

I.A.  Enterprise  Strategic 
Planning 

I.A.  1  -  Integration  of  loan  in  strategic 
planning  process 

lean  impacts  valuo  delivery  in  torms  of  cycle  time 
and  capability 

I.A. 2  -  Focus  on  customer  valuo  Customers  pull  value  from  enterprise  value  stroam 

I.A. 3  -  Lovoragirg  the  extended  enterprise  Valuo  stream  extends  from  customer  through  the 

enterprise  to  supporting  organizations 

Average 

I.B.  Adopt  Leon 

Paradigm 

I.B.l  -  learning  and  education  in  loan  for 
enterprise  loaders 

"Unlearning"  the  old,  learning  the  now 

I.B. 2  Sonior  monaqomonl  commitment 

Sonior  manaqomont  loadinq  it  personally 

I.B. 3  -  loan  enterprise  vision 

Now  mental  model  of  the  enterprise 

I.B. 4  -  A  sense  cf  urgonev 

The  primary  driving  forco  for  loan 

Average 

I.C.  Focus  on  Value 
Stream 

.C.1  -  Understanding  current  valuo  stroam  How  wo  now  deliver  valuo  to  customers 

.C.2  -  Enterprise  flow  "Single  piece  fow"  of  materials  and  information 

-C.3  -  Designing  future  valuo  stroam  Value  stream  t:>  meet  the  enterprise  visicr 

.C.4  -  Performance  moasuros _ | Performance  meosuret  drive  enterprise  behavior 

Average 

I.D.  Develop  Lean 
Structjre  and  Behavior 

.D.l  -  Entorpriso  organizational  orientation  Organize  to  support  value  delivery 

I.D. 2  -  Relationships  based  on  mutual  trust 

"Win-win"  vs.  'wc-thcy' 

.D.3  -  Open  and  timely  communications 

nformation  exchanged  when  roquirod 

I.D. 4  -  Employoe  ompowormont 

Decision-making  of  lowest  possible  lovol 

.D.5  -  Incentive  alignment 

Reward  the  behavior  you  want 

I.D. 6  Innovation  oncouroqomont 

From  risk  avorsion  to  risk  rowarcinq 

I.D. 7  -  Lean  chongo  agents 

Tho  inspiration  and  drivers  of  change 

_ Avamga 

Figure  14.  GLESAT  Summary  Sheet  (From  Lean  Enterprise  Initiative,  2005) 
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GOVERNMENT  LESAT  SECTION  I  SUMMARY  SHEET  -  CONTINUED 

Process  Definition:  Develop  and  deploy  lean  implementation  plans  throughout  the  enterprise  leading  to  (1)  long-term  sustainability,  (2) 
acquiring  competitive  advantage,  and  (3)  satisfaction  of  stakeholders. 


TTlLink 

Lean  Practice 

Loan  Characteristic 

Capability  Level 

Current 

Desired 

I.E.  Create/Refine  I.E.1  -  Enterprise-level  lean  transformation 

Transformation  Plan  p|on 

Charting  the  course  across  the  extended  enterprise 

I.E.2  -  Commit  resource*  for  loan 
improvements 

Resource  provision  for  loan 

I.E. 3  -  Provide  education  and  training 

Just-in-time  looming 

Average 

I.F.  Implement  Lean 
Initiatives 

I.F.1  -  Development  of  detailed  plans  based  Coordinating  lean  improvements 
on  enterprise  plan 

I.F.2  -  Tracking  detailed  implementation  Assessing  actuol  outcomes  ogoinst  gools 

Average 

I.G.  Focus  on  Continuous 
Improvement 

I.G.l  •  Structured  continuous  improvement 
processes 

Uniformity  in  how  we  get  better 

I.G.2  -  Monitoring  lean  progress 

Assessing  progress  toward  achiovmg  enterprise 
objectives 

I.G. 3  -  Nurturing  the  process 

Assure  senior  Icador  involvement 

1.0.4  -  Capturing  lessons  loomed 

Ensuring  that  successes  load  to  more  succossos 

I.G.5  •  Impacting  enterprise  strategic 
planning 

Results  load  to  strategic  opportunities 

Average 

Figure  14.  GLESAT  Summary  Sheel  (coni.) 
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SECTION  II  SUMMARY  SHEET  -  LIFE  CYCLE  PROCESSES 

Process  Definition:  Implement  lean  practices  across  life  cycle  processes  for  defining  customer  requirements,  designing  products  and  processes, 
managing  supply  chains,  producing  the  product,  distributing  product  and  services  and  providing  post  delivery  support. 


TTlLinlc  Leon  Practice  Leon  Characteristic  |  Current  Current 


LA.  Set-up  the 

Enterprise 

II.A.l  -  Lovorago  Loan  capability  for  now 

Exploiting  new  opportunities  arising  from  lean 
enablod  capabilities 

II.A.2  -  Optimize  the  capability  and 
_  utilization  of  assets _ 

Loan  enablos  mission  growth  through  the 

redapfeyment  of  assets _ 

II.A.3  -  Provido  capability  to  manage  risk, 
cost,  schodulo  and  performance 

Success  follows  offoctivc  nsk  management 

II.A.4  -  Allocate  resources  for  program/ 
project  development  efforts 

Teaming  for  success 

Average 

II.B.  Build 

Relationships 

II.B.1  -  Define  and  dovolop  relationships 

with  stakeholders _ 

Aligning  stakeholder  values  through  relationships 
that  build  credibility 

II.B.2  -  Optimize  the  relationship 

Creating  effective  relationships  to  achieve 
customer  value 

II.B. 3  -  Foster  innovation  and  knowlodgo- 
sharinq 

Incontivizing  innovation  through  stakeholder 
involvement 

Average 

Figure  14.  GLES AT  Summary  Sheet  (cont.) 


86 


SECTION  II  SUMMARY  SHEET  LIFE  CYCLE  PROCESSES  CONTINUED 

Process  Definition:  Implomont  loan  practice!  across  •  *f o  cycle  processes  for  defining  customer  requirement!,  designing  products  and  processes, 
monoqinq  supply  chains,  producing  tho  product,  distributing  product  and  services  and  providing  post  dolivory  support. 


TTl  Link 

II.C.  Develop  the  Plan 

Lean  Practice 

II.C.  1  -  Establish  a  requirement  definition 
process  to  optimize  lifo  cyclo 
valuo 

Lean  Characteristic 

Stakeholder  pull  vs  tochnology/product  push 

Capability  Level 

Current  Current 

II.C  2  -  Capturo  data  from  tho  extended 
enterprise  to  optimise  future 
rnq.iiromnnl  definitions 

Closed  loop  processes  are  in  place  to  capture 
operational  performance  data 

II.C. 3  -  Incorporate  noteholder  value  into 
design  of  products  and  pracessos 

Understanding  stakeholder  valuo  facilitates  fewor 
development  perturbations 

II.C. A  •  Incorporate  downstream 

itokcholder  values  into  products 
ond  processes 

Understanding  downstream  stakeholders  allows 

value  to  How  soamlossly 

II.C. 5  -  Create  a  multidisciplinary  approach 

Brooking  down  of  functional  silos  onobles  seamless 
rammunirntian  and  value  flow 

Average 

Figure  14.  GLESAT  Summary  Sheet  (cont.) 
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GOVERNMENT  LESAT  SECTION  II  SUMMARY  SHEET  -  CONTINUED 

Process  Definition:  Implement  loan  practices  across  life  cycle  processes  for  defining  customer  requirements,  designing  products  and  processes, 
managing  supply  chains,  producing  the  product,  distributing  product  and  services  and  providing  post  delivery  support. 


lean  Characteristic 

Capa 

TTL  Link 

Lean  Proctice 

Curren" 

' 

K 

II. D.  Implement 

the  Plan 

II.D.l  -  Utilize  knowledge  and 

capability  in  decision  making 

Strafogic  leveraging  of  stakeholder  capability 

II.D.2  -  Poster  lean  behavior 

throughout  tho  value  stroom 

Promoting  stakeholder  innovation  and  flexibility 

II. D. 3  -  Align  customer  requirements 
and  expectations  with  the 
exionded  enterprise 

Aligning  customer  and  stakeholder  expectations 

II. D. 4  -  Transition  product/sorvico  in 
a  loan  fashion 

Right  product  for  a  ready  customer 

Average 

IKE.  Learn, 

Improve,  and 

Sustain 

il.E.l  -  Enhance  value  of  delivered 
products  and  services  to 
customers  and  tho  enterprise 

Responding  to  the  voice  of  tho  customer 

II. E. 2  -  Provido  post  delivery  service, 
support  and  sustainability 

Providing  customer  solutions 

Average 

1 

Figure  14.  GLESAT  Summary  Sheet  (cont.) 
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SECTION  II  SUMMARY  SHEET  -  ENABLING  INFRASTRUCTURE 

Process  Uefinition:  To  ochiovo  a  successful  loan  transformation,  the  ontorpnso  infrastructure  must  support  tho  implementation  of  Loan  principles, 
practices  and  behavior. 


TTl  Link 

Lean  Practice 

Lean  Characteristic 

»  4 

III  A  Leon 

Organizational 

Enablors 

III.A.1  -  Finoncial  system  support* 
loan  transformation 

Lean  requires  appropriate  financial  data 

III.A.2  •  Enterprise  stakeholders  pull 
requirod  financial  infor¬ 
mation 

Uata  on  demand 

III  A3  -  Promulgate  tho  learning 
organization 

Looming  organizations  creato  a  Hoxiblo  workforce 

III.A.4  -  Enable  tho  loan  ontorpriso 
with  information  systems 
and  tools 

III.A.5  -  Integration  of  environmental 
protection,  health,  and 
safety  into  the  enterprise 

Facilitate  the  flow  of  information  and  knowledge 

"Cleonor,  hoalthier,  safor" 

Averooe 

III  B  lean  Process 

Enablers 

III . B.  1  Process  standardization 

Strive  for  consistency  and  re  u so 

III.B.2  -  Common  tools  and  systems 

Assurina  comaatibilitY.  roducina  costs 

III.B.3- Variation  reduction 

Reduce  uncertainty  by  reducing  variation 

Average 

_ 

Figure  14.  GLESAT  Summary  Sheel  (cont.) 
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E.  NAVAIR  ALIGNMENT  ASSESSMENT  TOOL  (NAAT) 

In  2005,  NAVAIR  contracted  with  GENESYS  Solutions  to  help  prepare 
and  administer  a  survey  to  all  employees  to  measure  alignment.  Senior  leadership,  i.e., 
the  Vice  Commander  and  the  head  of  Command  Staff,  championed  the  assessment.  In 
the  words  of  Vice  Admiral  Wally  Massenburg,  Commander,  NAVAIR  (Moore,  2005): 

...the  truly  great  organization  is  one  that  takes  the  time  to  look  at  itself 
with  a  critical  eye,  make  hard  changes,  and  improve  its  processes  to 
provide  the  best  support  to  its  customers  at  the  least  cost.  This  alignment 
assessment  is  a  chance  to  make  that  happen. 

Admiral  Mike  Mullen,  the  Chief  of  Naval  Operations,  says  (CNO,  2006), 

Alignment  is  the  degree  to  which  resources,  processes  and 
communications  support  vision  and  mission.  A  properly  aligned 
organization  can  accomplish  anything  it  attempts. 

NAVAIR  felt  that  an  alignment  assessment  was  necessary  to  confirm  that  the 
organization  was  doing  what  it  said  it  would  do,  and  to  make  sure  that  the  employees 
understand  the  leadership's  objectives.  They  wanted  meaningful  data  to  support 
continued  improvements  in  alignment  across  the  NAE.  They  felt  it  would  both  measure 
the  degree  of  alignment  and  help  create  alignment.  They  also  felt  that  the  questions  and 
structure  of  the  survey  would  clearly  communicate  expectations  to  the  workforce. 

The  NAAT  is  based  on  the  model  described  in  the  book,  “The  Power  of 
Alignment,”  by  Labovitz  and  Rosansky  (1997).  They  state  that  alignment  links  “strategy 
and  people  [by]  integrating  them  with  customers  and  process  improvement."  NAVAIR 
changed  their  model  slightly  with  four  areas  of  mission,  people,  processes,  and  “those  we 
serve”  (customers).  The  Mission  area  focuses  on  the  strategies  and  goals  of  the 
organization  as  defined  by  leadership.  The  People  area  focuses  on  the  management 
teams  and  staff  charged  with  executing  the  mission.  The  Processes  area  focuses  on  the 
methodologies  used  to  execute  the  mission.  Those  We  Serve  focuses  on  the  customers  or 
“beneficiaries”  of  the  mission.  Vertical  alignment  is  necessary'  to  show  the  extent  to 
which  People  can  successfully  execute  the  Mission.  The  horizontal  alignment  is 
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necessary  to  show  the  extent  to  which  each  business  Process  adds  value  to  Those  We 
Serve.  Figure  15  depicts  the  NAVAIR  alignment  model. 


Figure  15.  NAVAIR  Alignment  Model  (From  Ward  &  Tuck,  2005) 

Alignment  is  not  considered  to  be  a  ‘'normal"  state  in  any  organization. 
Measuring  alignment  is  needed  because  external,  constant  pressures  of  change  tend  to 
force  an  organization  out  of  alignment.  Changes  to  mission  are  driven  by  mandates, 
strategic  planning,  changing  threats,  alliances,  and  externally  imposed  issues.  Changes  to 
people  are  driven  by  organizational  demographics  and  mobility,  shifting  skill  sets,  and 
turnover.  Process  changes  are  driven  by  the  continual  need  to  re-engineer,  new 
technology,  and  cost.  Evolving  needs  drive  customer  changes. 

The  questions  used  in  this  assessment  model  were  developed  at  NAVAIR.  with 
support  from  GENESYS  contractors.  Leadership  was  involved  with  the  development  and 
grouping  of  the  questions,  and  they  also  suggested  topics  to  be  assessed. 

The  methodology  for  the  assessment  was  a  number  of  statements,  grouped  by 
factor,  and  assessed  on  a  seven-point  scale,  from  Strongly  Disagree  ( 1 )  to  Strongly  Agree 
(7).  The  seven-point  scale  was  then  convened  to  a  100-point  scale  for  reporting  purposes. 
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The  numerical  results  described  how  often  and  consistently  the  organizational  design  is 
followed. 


There  were  four  defined  dimensions,  based  on  the  alignment  model,  which 
devolved  into  20  factors  (based  on  NAVAIR  goals),  and  then  broken  into  114  total 
statements  to  be  answered.  There  were  two  open-ended  questions  to  answer.  The  four 
dimensions  and  20  factors  were  organized  as  shown  in  Table  6. 


Table  6.  Assessment  Dimensions  and  Factors 


Dimensions 

Factors 

Mission 

•  Current  Readiness 

•  Future  Readiness 

•  Cost  of  Doing  Business 

•  Agility 

•  Enterprise  Alignment 

•  Metrics 

People 

•  Performance  Management 

•  Local  Leadership 

•  National  Leadership 

•  Supervision 

•  Workforce  Shaping 

Process 

•  Competency  Alignment 

•  Naval  Aviation  Enterprise 

•  Virtual  SYCOM 

•  Human  Capital  Strategy 

•  AIRSpeed 

Customer 

•  Customer  Focus 

•  Partnerships 

•  Responsiveness  to  Fleet 

The  results  from  the  2005  survey  showed  that  61  percent  of  NAVAIR  personnel 
participated  in  the  survey  (Ward  &  Tuck,  2005).  The  overall  alignment  rating  was  64.6 
percent,  which  was  considered  average  for  an  organization  beginning  this  process  and 
consistent  with  first  measurements  across  other  Navy  commands.  Action  plans  can  be 
developed  from  the  results,  depending  on  the  analysis  done.  The  2006  survey  utilized  the 
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same  statements,  which  (when  analyzed)  should  help  provide  a  trend  line  and 
opportunities  for  improvement. 

Also,  in  the  2005  survey,  there  was  a  four  to  eight  percent  rating  given  to  the 
command's  capability  to  execute  initiatives  such  as  NAE,  AIR Speed,  human  capital 
strategy,  and  the  “virtual  SYSCOM."  The  low  percentage  was  perceived  to  reflect  the 
early  stages  of  the  initiatives.  Also,  it  is  believed  that  on  a  case-by-case  basis,  higher 
levels  of  capability  exist,  especially  among  senior  levels  of  the  organization. 

The  survey’s  message  was  that  NAVAIR  needs  to  continue  to  build  stronger 
alliances  with  Fleet  customers,  improve  the  communication  process  within  the  command, 
promote  innovative  thinking,  develop  better  metrics,  and  collect  the  right  data.  Senior 
leadership  and  supervisors  were  encouraged  to  engage  more  in  direct  interaction  with  the 
workforce  rather  than  "broadcasting”  when  sharing  information. 

Within  the  rest  of  NAE,  specifically  Naval  Air  Forces,  the  survey  has,  to  date, 
been  conducted  only  once.  When  the  results  from  the  Naval  Air  Forces  survey  were 
analyzed  and  compiled,  they  showed  that  the  organization's  alignment  around  Current 
Readiness  was  identified  as  needing  improvement  (Ward  &  Tuck,  2005).  Drilling  down 
further  into  the  analysis,  the  key  barrier  was  identified  as  a  lack  of  innovative  thinking. 
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IV.  ASSESSMENT  TOOLS  AND  INTERVIEWS  ANALYSES,  AND 
INTERNAL  ORGANIZATIONAL  ENVIRONMENT 

A.  INTRODUCTION 

This  chapter  draws  from  our  literature  review  and  interviews  and  analyzes  the 
following  four  self-assessment  tools:  Department  of  Defense.  Continuous  Process 
Improvement  Transformation  Guidebook  (DoD  CPITG);  Navy  Performance  Excellence 
Guidebook  (NPEG);  Lean  Aerospace  Initiative,  Government  Lean  Self-Assessment  Tool 
(LAI  GLESAT);  and  NAVAIR  Alignment  Assessment  Tool  (NAAT).  We  discuss  the 
extent  to  which  the  various  tools  can  provide  value  to  the  AIR  Speed  Merge  initiative.  We 
provide  interview  results  and  summarize  key  learning  points. 

Our  analysis  includes  an  overview  of  the  current  environment  within  the  Naval 
Aviation  Enterprise  (NAE),  using  a  systems  construct  as  our  theoretical  foundation 
(Senge,  1990).  Systems  theory'  posits  that  the  extent  to  which  an  organization's  direction 
(mission,  goals,  strategy)  fits  or  is  congruent  with  its  external  and  internal  environments, 
better  performance  results.  The  internal  environment  is  framed  from  our  individual 
viewpoints  within  the  NAE:  the  Commander,  Naval  Air  Forces  Staff.  San  Diego,  CA, 
and  the  Naval  Air  Warfare  Center,  Weapons  Division.  China  Lake,  CA. 

Finally,  we  provide  our  assessment  of  the  four  assessment  tools  by  comparing 
their  structure,  how  they  relate  to  the  key  concepts  outlined  in  Chapter  2,  and  their 
positive  and  negative  aspects  against  particular  criteria. 

B.  ENVIRONMENTAL  ASSESSMENT 

As  we  evaluate  the  four  enterprise  assessment  tools  against  the  needs  of 
AIRSpecd  Merge  initiative,  it  is  essential  that  we  have  a  sense  of  environmental  forces, 
trends  and  capabilities  in  the  Naval  Aviation  Enterprise  (NAE).  Knowledge  of  the  NAE 
internal  environment  is  therefore  an  interdependent  variable  in  terms  of  anticipating  how 
well  a  particular  tool  or  methodology  is  likely  to  fit  or  become  aligned  with  the 
organizational  design  (structure  and  processes). 
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Part  of  this  “environment"  includes  the  objectives  and  plans  of  internal  and 
external  stakeholders  in  the  following  commands: 

•  Commander,  Naval  Air  Forces  (CNAF)  (NAE  Chief  Executive  Officer) 

•  Commander,  Naval  Air  Systems  Command  (COMNAVAIRSYSCOM) 
(NAE  Chief  Operating  Officer) 

•  Deputy  Commandant  for  Aviation,  1  leadquarters  Marine  Corps  (1 1QMC) 

•  Commander,  Naval  Air  Forces,  U.S.  Atlantic  Fleet  (CNAL) 

•  Chief  of  Naval  Air  Training  (CNATRA) 

•  Director,  Air  Warfare  Division  (OPNAV  N8X)  (NAE  Chief  Financial 
Officer) 

•  Director,  Fleet  Readiness  Division  (OPNAV  N43) 

Due  to  time  and  space  limitations,  we  will  consider  environmental  assessments 
for  Naval  Air  Systems  Command  (NAVAIR)  and  Commander,  Naval  Air  Forces 
(CNAF)  as  generally  representative  of  the  overall  NAE  environment. 

It  appears  to  be  common  knowledge  that  the  overall  viability  and  culture  of  an 
enterprise  is  shaped  by  its  communication  structure,  i.e.,  the  ability  of  leaders  and 
managers  to  clearly  transmit  both  organizational  direction  and  operations,  in  terms  of 
strategies,  goals  and  objectives,  and  rewards.  Other  non-strategy  focused,  uncoordinated, 
and  unfiltered  communications  can,  and  often  do,  create  unintended  "noise.”  Various 
components  of  distracting  noise  may  be  relevant  and/or  useful  in  their  own  right,  but  if, 
where  the  structure  to  clarify  and  dedicate  scarce  resources  towards  organizational 
priorities  is  flawed,  employees  are  forced  to  react  to  the  loudest  problem  and  to  compete 
for  resources.  In  summary,  communication  of  too  many  initiatives,  constantly  changing 
initiatives,  and  pervasive  admonitions  from  external  and  internal  sources  that  everything 
is  urgent  and  important  translates  into  scattered  focus  and  poor  performance.  Managers 
and  employees  become  confused,  rework  increases,  and  workers  bounce  from  critical 
issue  to  critical  issue,  much  like  fire  fighters.  The  “signal  to  noise"  ratio  may  be  high  in 
the  enterprise  under  study. 

What  follows  is  a  sample  of  the  number  of  initiatives  or  "noise"  that  the  current 
NAVAIR  and  CNAF  management,  staff,  and  workforce  are  receiving. 
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1.  NAVAIR  Environment 

In  the  current  environment,  there  are  several  initiatives  in  place  at  NAVAIR 
Headquarters  and  at  the  business  unit  level  that  require  time,  energy,  personnel  expertise, 
facilities  and  dollars  -  in  short,  numerous  priorities.  Managers  try  to  juggle  resources  and 
shield  employees  from  excessive  noise,  but  on  net,  our  assessment  is  that  many  workers 
perceive  the  amount  of  activity  as  “chaotic.”  A  few  of  those  initiatives  are  described 
below: 


a.  Base  Realignment  and  Closure  (BRAC) 

In  2005,  a  decision  was  made  by  Congress  as  part  of  the  BRAC  process  to 
move  all  weapons  and  armament  work  to  China  Lake,  CA,  and  form  the  Navy  Weapons 
Center  of  Excellence.  In  the  process,  there  are  no  actual  bases  being  closed,  but  weapons 
workload  is  being  moved  from  seven  DoD  sites  to  China  Lake.  However,  it  is  now  a 
year  later,  and  the  work  being  moved  is  still  undefined.  This  is  a  disturbing  change 
affecting  peoples'  lives  and  work.  The  unit  has  suffered  considerable  loss  of  personnel  in 
the  last  year  because  of  the  BRAC  decision.  The  “mental  noise"  surrounding  the  BRAC 
decision  acts  as  a  filter  or  impediment,  preventing  employees  from  being  genuinely 
involved,  learning  about  and  implementing  lean  activities. 

b.  Enterprise  Resource  Planning  (ERP) 

NAVAIR  implemented  ERP  for  financial  and  human  resource  processes 
over  three  years  ago.  Since  then,  we  rely  upon  it  for  timecards,  training  registration,  and 
for  financial  tracking.  The  system  has  not  achieved  what  was  expected.  The  Navy  has 
four  pilot  ERP  implementations  that  are  merging  within  the  next  two  years.  NAVAIR  is 
once  again  taking  the  lead  to  implement  the  merged  ERP  first.  With  “merged  ERP,”  we 
will  have  our  processes  changed  again  and  that  transition  is  not  expected  to  be  easy,  again 
introducing  competition  to  Lean  transformation. 

c.  National  Security  Personnel  System  (NS PS) 

NSPS  is  the  new  DoD  personnel  system,  which  is  changing  performance 
management,  pay,  and  promotion  processes.  Besides  apparently  being  strongly  contested 
by  the  unions,  part  of  the  workforce  is  in  NSPS  and  the  other  part  is  spread  across  two 
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other  performance  management  systems,  once  again  creating  noise  and  distraction  for 
managers  and  employees. 

d.  Total  Force  Readiness  (TFR) 

TFR  was  previously  called,  “Human  Capital  Strategy.”  The  Navy  in 
general,  and  NAVAIR  specifically,  is  focused  on  determining  and  adjusting  the  quantity 
and  quality  of  military  and  civilian  employees,  including  the  array  of  needed  skills, 
emerging  new  products  and  services,  and  planning  for  the  long-war.  There  are  conflicting 
data  calls  within  Headquarters,  which  results  in  increased  business  unit  tasking,  which 
demands  urgent  and  complex  deliverables. 

e.  Leadership  Change 

In  December  of  2006,  NAVAIR  senior  leadership  is  changing.  The 
Commander  is  retiring,  and  a  new  Commander  will  replace  him  with  a  new  style  of 
leadership  and  focus,  including  a  new  Vice  Commander  in  2006.  Past  experience  dictates 
that  the  new  leadership  will  attempt  to  make  its  mark  by  introducing  greater  change  and 
accompanying  urgency.  Systems  thinking  and  acting  describes  this  type  of  phenomenon 
as  a  vicious  cycle  (Senge,  1990).  The  answer  involves  intervention  in  the  cycle  to 
prevent  further  erosion,  i.e.,  prioritized  ranking  of  issues  and  involvement  and 
commitment  from  the  top. 

2.  CNAF  Environment 

The  CNAF  organization,  as  part  of  the  NAE,  is  a  focal  point  for  many  of  the 
cultural,  organizational,  and  process  changes  in  the  DoD  and  the  Navy.  In  addition  to 
externally-driven  CNAF  initiatives,  there  are  also  internally  influenced  changes.  Again, 
our  net  assessment  appears  to  revolve  around  the  communication  structure,  i.e.,  the  extent 
to  which  CNAF  personnel  and  subordinate  commands  understand  their  priorities, 
including  a  process  to  reduce  unwanted  noise,  will  be  related  to  successful  performance. 
Several  examples  of  competing  initiatives  are  described  below: 
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a.  Defense  Readiness  Reporting  System  ( DRRS ) 

The  DoD  capabilities  based  readiness  reporting  system,  which  will 
fundamentally  change  the  process  of  valuing  and  reporting  force  readiness.  The  Navy 
previously  measured  readiness  at  the  unit  (squadron)  level,  across  broad  mission  areas,  by 
using  the  Status  of  Resources  and  Training  System  (SORTS)  to  assess  the  amount  and 
condition  of  assigned  resources.  The  new  DRRS  system  mandates  readiness  reports  based 
on  unit  and  organizational  capabilities,  which  are  defined  by  specific  performance  criteria 
for  assigned  missions,  tasks,  conditions,  and  standards.  The  transition  could  be  compared 
to  US  Steel  developing  new  accounting  procedures  for  its  year-end  reports,  in  a  new 
language,  which  does  not  use  money  as  the  unit  of  measure.  Each  aircraft  community  in 
the  Navy  is  now  in  the  process  of  redesigning  its  readiness  reporting  system  to  provide 
capabilities  based,  readiness  reporting  to  comply  with  DRRS. 

b.  Board  of  Directors  (BOD)  and  Cross  Functional  Teams  (CFT) 

The  NAE  is  developing  its  processes  and  is  organized  into  a  Board  of 

Directors  (BOD)  consisting  of  Naval  Aviation's  senior  leaders  and  three  Cross 
Functional  Teams  (CFT)  (Moore,  2005): 

1.  Naval  Aviation  Readiness  Integrated  Improvement  Program 
(NAVRIIP)  CFT,  led  by  Commander,  Naval  Air  Force,  U.S. 
Atlantic  Fleet 

2.  Cost  Management  CFT,  chaired  by  Mr.  Garry  Newton,  OPNAV, 
Director  of  Air  Warfare  (N88)  and  Director  Fleet  Readiness  (N43). 

3.  Total  Force  Readiness,  led  by  RADM  Shannon,  Director  of  NAE 
Total  Force  Readiness 

Additional  structures  are  provided  under  each  CFT  to  manage  specific 
tasking  from  the  top  level  CFT.  Decisions  and  actions  that  can  not  be  easily  resolved  are 
tasked  to  Barrier  Removal  Teams  (BRT).  Both  CFTs  and  BRTs  form  and  operate  from 
outside  of  the  command  structures  from  which  they  draw  their  members.  The  CNAF 
personnel  leading,  or  assigned  to  a  CFT  or  BRT,  must  still,  however,  perform  the  primary 
command  jobs  to  which  they  have  been  ordered.  Needless  to  say,  the  competing  demands 
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involved  in  lasking  and  prioritization  of  work  between  the  CFT/BRT,  and  the 
requirements  of  command  leadership  can.  and  does,  generate  substantial  confusion  and 
"noise."  Matrix  organizations  can  experience  the  same  conflict  between  project 
management  teams  and  the  competency  functions  within  the  same  command. 

c.  Fleet  Response  Plan  ( FRP ) 

The  Fleet  Response  Plan  (FRP)  is  a  flexible  U.  S.  Navy  operational 
concept  that  is  designed  to  more  rapidly  develop  and  sustain  readiness  in  ships  and 
squadrons.  In  the  case  of  a  national  crisis  or  contingency  operation,  the  Navy  can  then 
quickly  surge  significant  combat  power  to  the  scene.  FRP  changes  the  way  we  operate, 
train,  man,  and  maintain  the  fleet.  CNAF  has  the  responsibility  of  manning,  equipping, 
and  training  Naval  Aviation  forces.  Previously  these  functions  were  based  on  an  18- 
month  readiness  cycle.  The  advent  of  FRP  and  its  flexibility  puts  CNAF  in  the  position  of 
making  long-range  planning  decisions  based  on  greater  uncertainty. 

d.  Nava I  Aviation  Readiness  Integrated  Improvement  Program 
(NA  VRIIP) 

The  NAVRIIP  is  a  management  structure  for  readiness  and  cost  drivers.  It 
is  a  NAE  CFT.  The  goal  is  to  provide  aircraft  ready  for  tasking  at  reduced  cost.  The 
business  rules  and  metrics,  on  which  readiness  and  cost  are  measured,  cut  across 
command  structures  and  can  create  discontinuity  in  focus  and  budget  processes  at  the  unit 
level  if  not  aligned  to  local  processes. 

e.  Staff  Reorganization 

CNAF  evolved  from  a  staff  reorganization  of  Commander,  Naval  Air 
Force,  U.S.  Pacific  Fleet  and  Commander,  Naval  Air  Force,  U.S.  Atlantic  Fleet,  each 
with  its  own  cultures,  policies,  procedures,  and  leadership  styles.  CNAF,  in  its  current 
state,  is  one  bi-coastal  command  with  some  functional  codes  split  between  coasts.  While 
decisions  and  policies  are  made  collectively,  the  CNAF  command  structure  remains 
divided  by  coast  with  some  remaining  legacy  processes  and  policy.  The  discontinuity 
between  newly-formed  command  structures  and  the  legacy  environment  is  at  the  heart  of 
needing  improved  vertical  and  horizontal  communications. 
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C.  INTERVIEWS 

1.  Interviews  Provided  Qualitative  Data 

Approximately  10  semi-structured  interviews  were  conducted  to  develop  a  deeper 
understanding  of  enterprise  assessment  concepts  and  to  collect  information  on  the 
specific  use  of  the  various  assessment  tools  discussed  in  this  thesis.  The  process  allowed 
for  secondary  pursuit  of  topical  information  and  broadly  applicable  insight,  by  asking 
follow-on  questions  and  probing  for  previously  unforeseen  issues. 

An  additional  driver  for  a  qualitative  approach  was  the  fact  that  the  DoD  CPITG 
and  NPEG  tools  have  recently  been  signed  or  submitted  for  signature,  respectively. 
Because  of  their  infancy  as  CPI  tools  and  processes,  there  were  no  implementations  that 
could  be  measured  to  compare  against  the  LESAT/GLESAT  and  NAAT.  Most  of  the 
respondents  were  experienced  with  LESAT/GLESAT  or  internally-developed  enterprise 
assessments.  The  NAAT  was  administered  to  the  authors  of  this  thesis,  through  the 
course  of  their  command  assessments,  allowing  direct  reporting  on  its  use  and 
implementation. 

2.  Interview  Candidates 

The  individuals  interviewed  represented  a  multi-service  and  aerospace  industry 
view  of  enterprise  assessment,  across  senior  to  middle  management  levels  of  their 
organizations.  Interviewees  were  chosen  based  on  their  experience  in  lean  transformation 
and  in  using  one  of  the  assessment  tools  being  researched.  They  all  had  direct  experience 
with  the  LAI  LESAT  or  GLESAT  and  their  answers  are  based  on  the  use  of  that  tool.  The 
DoD  CPITCi  and  NPEG  tools  are  new;  therefore  no  one  was  interviewed  who  had  direct 
experience  with  these  tools.  Consequently,  interview'ees  were  not  selected  for  specific 
tool  usage,  rather  for  their  experience  with  Lean  assessments  and  their  use  of  other  Lean 
assessment  tools. 

The  following  w'ere  interviewed: 

•  Six  individuals  from  three  large  aerospace  firms  from  middle  level 
management  up  to  a  business  area  vice  president 
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•  Two  individuals  from  the  Air  Force,  one  active  duty  officer  and  one 
government  civilian 

•  One  active  duty  officer  from  a  Naval  Aviation  Depot 

•  A  senior  Navy  civilian  from  NAVAIR 

3.  Interview  Questions 

The  interview  questions  were  developed  subsequent  to  the  development  of  the 
Chapter  2  key  concepts  for  enterprise  assessment.  The  tools  and  our  ability  to  evaluate 
surrounding  business  processes  were  then  assessed  based  on  a  consistent  set  of  criteria. 

The  responses  to  each  question  were  summarized  to  focus  on  the  key  observations 
given  after  each  stated  question.  The  purpose  of  this  approach  was  to  ensure  that  essential 
issues  could  be  fully  described  without  direct  attribution.  To  do  otherwise  would  have 
violated  the  confidentiality  of  proprietary  business  processes  used  to  deliver  value  to  their 
customers. 


a.  Was  there  a  particular  event  or  “burning  platform  "  that  led  your 
company  to  do  Lean  and  Enterprise  Assessments? 

Three  processes  emerged  as  the  most  common  stimuli  for  adopting  a  lean 

methodology  of  continuous  process  improvement  and  enterprise  assessment: 

•  Direction  from  company  leadership  with  the  authority  to  enforce  the 
activity  and  set  an  enterprise  vision. 

•  An  internal  success  story  with  dramatic  improvements  that  made  the 
success  clearly  evident  to  the  surrounding  organization. 

•  Compliance  with  some  external  influence  such  as  customer  contractual 
requirements,  market  forces  that  increased  competition  and  reduced 
revenue,  or  a  pending  budget  reduction  or  termination. 

The  leadership  push  for  enterprise  lean  processes  and  assessment 
generally  came  from  observation  of  successful  lean  practices  either  done  externally,  or 
isolated  cases  internally.  In  all  the  responses  there  was  a  visible  stimulus  for  change  or 
visionary  direction  by  senior  leadership. 
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The  preexisting  leadership  philosophy  and  culture  in  the  organization,  and 
its  tendencies  for  continuous  process  improvement,  defined  the  level  and  length  of 
stimulus  required  for  change.  In  all  interview  cases,  without  leadership  involvement  there 
was  no  ability  to  graduate  to  enterprise  CPI  and  utilize  assessment  processes.  The  general 
consensus  seemed  to  be  that  business  units  would  accept  CPI  functions  for  their  benefit, 
but  had  no  time  for  enterprise  assessments,  as  they  could  not  see  the  value  in  their  daily 
work.  One  senior  official  stated,  "Without  the  boss's  support  for  implementation  of 
assessments,  I  would  have  been  fired  in  the  first  three  days!" 

b.  What  too I  did  you  use  to  do  your  assessment? 

Only  two  of  the  four  tools  being  considered  in  this  thesis  are  being  used  in 
government  or  industry:  the  LAI  LESAT  (with  GLESAT  variation)  and  the  NAAT. 
Beyond  the  tools  being  considered  for  comparison  in  this  document,  there  are  other 
internally-developed  assessment  tools  and  processes  that  are  being  used  by  industry.  In 
some  instances  these  internally-developed  assessment  tools  were  built  from  systems 
already  resident  in  an  organization.  In  other  instances,  they  were  built  from  scratch  using 
current  quality  programs  and  maturity  models  as  their  core  structure. 

c.  What  process  did  you  use  to  choose  a  too I  for  assessment? 

The  consistent  response  to  this  question  was  that  there  was  not  a  process 
of  selection.  The  assessment  processes  utilized  were  either  directed  by  leadership  with  an 
organizational  connection  to  LAl's  LESAT/GLESAT,  resulted  from  an  internal 
development  project,  or  in  one  case,  the  tool  was  recommended  by  "word  of  mouth." 
While  there  was  one  instance  of  using  an  existing  tool  and  then  switching  to  an  internally 
developed  program,  the  process  was  not  designed  as  a  decision-making  process  for 
assessment  selection,  it  was  simply  trial  and  error. 

d.  How  did  you  scope  the  use  of  the  tool:  At  the  enterprise  level,  or 
did  you  start  with  small  group? 

The  implementation  of  the  assessments  that  were  used  was  generally 
scoped  to  major  programs  or  lines  of  business,  which  tended  to  be  smaller  groups  in 
relation  to  the  larger  enterprise.  The  industry  participants  indicated  that  one  reason  for 
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limiting  the  scope  to  the  business  unit  was  a  concern  that  above  this  level  it  would  be 
hard  to  generate  actionable  data  that  would  have  an  effect  on  the  bottom  line.  The 
government  participants  also  indicated  that  the  assessments  were  utilized  at  a  functional 
business  level. 


e.  What  process  did  you  use  to  administer  the  tool  and  how  was  the 

assessment  recei red? 

The  method  for  implementing  an  assessment  was  completely  dependent 
on  the  workforce's  familiarity  with  the  language  of  the  assessment  and  the  process  for 
collecting  the  data.  In  all  cases  but  one,  the  first  implementation  required  full  facilitation 
by  trained  experts.  Often  the  individuals  involved  in  this  process  came  out  of  the  quality 
assurance  functions  within  a  company  and  would  be  augmented,  or  even  led  by  authors 
of  the  assessment  tool,  as  outside  experts. 

It  was  commonly  noted  that  the  facilitated  events  produced  better  results. 
First,  because  the  observation  of  a  third  party,  sometimes  in  an  off-site  setting,  provided 
oversight  to  ensure  a  quality  input.  Secondly,  the  facilitator  was  there  to  help  teach  and 
inform  when  the  participants  had  confusion  over  the  intent  of  a  question,  or  section,  of 
the  assessment.  One  industry  interviewee  noted  that  the  use  of  recognized  experts  in  the 
field  of  CPI  such  as  certified  Six  Sigma  Black  Belts  lent  credibility  to  the  process. 

The  amount  of  time  dedicated  to  a  fully  facilitated  assessment  was  at  most 
two  days  at  the  program  office  level.  In  one  instance,  the  assessment  was  delivered  as 
pail  of  a  program  review  meeting  because  all  the  required  participants  were  already 
assembled.  Where  the  senior  leadership  was  involved,  generally  one  half-day  was  all  that 
could  be  spared  from  busy  schedules.  In  one  case,  a  web-based  presentation  of  the 
assessment  was  created  to  allow  individuals  to  complete  the  work  on  their  own  schedule. 
In  another  case  the  initial  assessment  was  completed  using  data  extracted  from  other 
quality  programs  in  an  attempt  to  save  time  and  reuse  previously  acquired  information. 

The  remaining  issue  for  implementation  of  an  enterprise  assessment  is 
data  reduction,  analysis,  and  feedback.  All  respondents  reported  that  there  is  a  balance 
between  speed  and  quality  that  must  be  found.  Generally,  the  best  approach  seemed  to  be 
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a  quick-look  qualitative  response  (most  tools  have  a  fast  tabulation  capability)  followed 
by  a  formal  debrief  with  both  quantitative,  and  qualitative,  analysis  of  the  findings. 

f.  Was  the  complexity  of  the  assessment  tool  worth  the  return  to 
your  organization  ? 

In  circumstances  where  the  assessment  process  was  dictated  by  leadership, 
there  was  little  or  no  attempt  to  justify  the  value  of  assessment.  One  of  the  mid-level 
managers  interviewed  remarked,  “When  the  boss  says  do  it,  your  return  on  investment  is 
not  an  issue."  In  the  cases  where  assessments  were  being  used  as  part  of  an  overall  CPI 
strategy  or  incentive  program,  the  complexity  of  the  process  was  reported  to  be  worth  the 
effort  because  the  resultant  learning  and  improvement  could  be  seen  and  trusted  by  the 
organization.  In  one  case,  the  company  trusted  its  processes  so  implicitly  that  project  bids 
were  based  on  the  knowledge  that  lean  and  CPI  would  achieve  the  increase  in  efficiency 
and  effectiveness  necessary  to  meet  future  expectations  of  lower  priced  bids.  In  another 
case,  annual  performance  targets  and  strategies  resulted  from  the  assessment  process,  and 
lean  targets  were  included  in  the  annual  performance  evaluation  process  for  individuals. 

g.  Did  the  assessment  require  a  dedicated  team?  Is  the  team 
permanently  assigned? 

The  consistent  response  across  all  interviewees  was  that  a  dedicated 
assessment  team  was  required,  with  some  portion  of  the  personnel  permanently  assigned 
to  manage  the  overall  CPI  process.  The  common  perspective  was  that  the  combination  of 
full  time  support  and  a  consistent  message  from  leadership  maximized  the  authority  of 
the  assessment  team. 

In  some  instances  local  agents  were  utilized  to  augment  the  permanently 
assigned  assessment  team.  One  senior  manager  said,  "If  you  don't  have  a  technically 
adept  person  who  lives  and  breathes  it  for  the  organization,  it  will  fall  short."  Another 
tactic  that  was  utilized  to  facilitate  implementation  of  an  enterprise  assessment  was 
assignment  of  an  outside  third  party  facilitator  with  expertise  in  CPI.  In  this  case  an 
industry  respondent  utilized  a  senior  Black  Belt  certified  employee  respected  in  another 
business  sector  to  lend  his  name  and  reputation  to  the  assessment  process. 
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h.  Have  you  used  the  too I  more  than  once  to  measure  your 
progress? 

In  most  cases  the  assessments  were  conducted  more  than  once.  In  a  couple 
of  instances,  the  leadership  transitioned  between  assessments.  When  the  new  leader  did 
not  place  emphasis  on  the  assessment,  it  fell  into  disuse.  The  key  driver  for  completion  of 
the  assessment  at  the  business  unit  level  was  the  degree  to  which  the  data  and  metrics 
from  the  assessment  were  embedded  in  the  daily  business  processes. 

Where  the  assessment  was  imbedded  in  either  the  normal  strategy 
development  process,  part  of  long-range  planning,  or  included  in  the  CPI  processes,  it 
was  repeated  on  an  annual  or  semi-annual  basis.  In  some  instances  where  feedback  or 
resulting  action  from  an  assessment  was  delayed,  the  assessment  would  fall  out  of  the 
normal  business  cycle,  in  one  case  the  company  is  considering  a  bi-annual  assessment 
process. 


i.  Is  your  company  using  more  than  one  tool  to  assess  your  lean 
activities? 

In  most  cases  the  organizations  only  used  one  enterprise  assessment  tool 
and  process  due  to  the  investment  of  time  and  resources  to  train  the  workforce  on  the 
logic  and  language  of  the  chosen  tool.  It  was  usually  difficult  enough  to  gain  acceptance 
for  one  tool,  much  less  separate  tools. 

j.  How  involved  was  leadership  in  the  assessment(s)  direction, 
analysis,  and  communicating  the  results? 

In  all  cases  leadership  was  stated  as  a  requirement  to  ensure  compliance 
with  the  assessment  process.  Most  of  the  preceding  questions  also  established  the 
importance  of  a  leadership  presence  in  the  opinion  of  the  interviewees.  It  was  noted  by 
half  of  the  respondents  that  as  the  leadership's  resolve  weakened,  its  ability  to  enforce 
policy,  and  consequently  the  enterprise  assessment  process,  was  also  weakened. 

On  the  opposite  end  of  the  spectrum,  where  there  was  strong  leadership, 
the  organizations  remained  focused  on  conducting  the  assessment  and  using  the  results  in 
its  business  process.  In  one  case  there  was  a  punitive  termination  of  a  senior  employee 
because  he  refused  to  institute  a  lean  process  prescribed  by  the  assessments. 
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4.  Interview  Themes 

A  number  of  key  points,  that  were  common  to  all  the  interviewees,  need  to  be 
addressed  to  gain  a  more  complete  view  of  the  implementation  of  enterprise  assessment. 

•  Enterprise  assessments  were  generally  connected  to  other  strategic 
enterprise  functions  for  CPI  such  as  strategic  planning,  supply  chain 
management,  and  Value  Stream  Mapping. 

•  Where  the  enterprise  assessment  tools  were  applied  to  more  narrow 
business  unit  functions,  the  fact  that  the  assessment  process  was  tied  to  a 
larger  CPI  process  kept  them  applicable  across  the  broader  enterprise. 

•  In  many  cases,  the  challenge  of  using  an  externally-developed  assessment 
tool  was  interpreting  its  lexicon.  If  the  terminology  was  not  consistent 
with  the  organization,  there  was  a  great  deal  of  familiarization  and  training 
required  before  participants  could  make  sense  of  the  tool.  Facilitation  was 
the  used  to  mitigate  this  conflict. 

•  It  was  noted  that  the  use  of  local  agents  for  CPI  and  coordination  of 
assessment  issues  was  more  efficient  because  they  managed  the  daily 
alignment  of  assessment  processes.  The  local  agent  also  is  a  key  individual 
for  managing  early  adoption  and  training  of  a  new  assessment  initiative. 

•  The  influence  of  an  outside  observer  tended  to  mitigate  some  of  the  rivalry 
and  intra-departmental  power  conflicts  that  can  become  apparent  in  an 
assessment  process. 

•  In  two  cases  where  local  agents  argued  for  internal  tailoring  of  LESAT, 
the  leadership  denied  their  request  with  the  goal  of  eliminating  variation  in 
the  collected  data  induced  by  changing  assessment  structure  over  time. 

•  In  both  the  civilian  and  government  sectors,  it  was  noted  that  frequent 
turnover  of  leadership  and  employees  is  a  barrier  to  effective  enterprise 
assessment  and  behavior. 
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D. 


ANALYSIS  OF  SELF-ASSESSMENT  TOOLS 


In  analyzing  each  tool,  we  kept  in  mind  the  research  questions  and  the  value  each 
tool  could  provide  to  AIRSpeed's  progress  and  maturity.  Each  tool  needed  to  be  able  to 
measure  the  “lean"  maturity  of  the  enterprise  and  enable  implementation  of  the  key 
concepts.  We  used  these  factors  as  a  baseline  for  our  analyses.  We  evaluated  the  “pros 
and  cons”  of  each  tool  within  the  individual  analysis  area. 

To  ensure  that  we  were  aligned  with  our  stated  thesis  goals,  we  did  an  analysis  of 
the  AIR Speed  concepts,  goals,  and  objectives.  From  those,  we  compiled  a  new  set  of 
AIR  Speed  key  concepts  to  compare  against  the  key  concepts  derived  from  the  literature 
review.  This  created  a  bridge  from  the  internal  NAE  communication  to  the  literature 
review.  The  AIRSpeed  Key  Concepts  and  our  Assessment  Key  Concepts  from  the 
literature  review  are  compared  in  Table  7. 
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Table  7.  Comparison  of  AIR  Speed  Key  Concepts  and  Assessment  Key  Concepts 


Vision 

The  NAE  overall  vision  is  the  enterprise  target  to  focus  the  thought  and  actions  of  all 
personnel:  To  deliver  the  right  force,  with  the  right  readiness,  at  the  right  cost,  at  the  right 

time  -  today,  and  in  the  future. 

Strategic  Goals 

•  Balance  current  and  future  readiness 

•  Reduce  the  cost  of  doing  business 

•  Improve  agility 

•  Ensure  alignment 

•  Implement  fleet  driven  metrics 

AYRSpeed  Key  concepts 

Assessment  Key  Concepts 

• 

Decrease  inventory 

•  Knowledge  management,  change 

• 

Decrease  cycle  time 

management,  and  culture 

• 

Improve  reliability 

•  Systems  thinking  perspective 

• 

Decrease  operating  expenses 

•  Well  designed,  mature  processes 

• 

Strategic  linkages  of  CPI  projects 

•  Value  for  employees  and  partners 

• 

Standard  set  of  tools,  applied 

•  Differentiates  between  a  product  / 

effectively 

service  and  Fleet  Readiness  Center 

• 

Communication  and  change 

(manufacturing)  environment 

management 

•  Incorporates  lean  concepts 

• 

Manage  key  interfaces  across 

•  Fit  -  does  the  organization  have  the 

industry,  Department  of  Defense, 

time  and  resources  to  conduct  the 

and  academia 

assessment 

• 

Alignment  of  metrics  and 

•  Measures  provide  actionable 

supporting  data  systems 

information  (maturity  level 'DMAIC) 

• 

Change  agents 

•  Promotes  alignment  -  AIR Speed  as  a 

• 

Increasing  productivity 

corporate  leadership  value 

• 

Uses  systems-of-systems 

•  Waste-wise  readiness,  not  readiness 
with  disregard  for  waste 

• 

Active  leadership  involvement 

• 

Promotes  learning  by  building 
new  skills  in  a  focused  manner 

•  Rights  to  assessment  product  are 
wholly  owned  by  the  enterprise 

•  Is  the  enterprise  “signal-to-noise"  ratio 
measured  for  effective  communication 
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1.  Comparison  of  Self-Assessment  Tools 

Table  8  contains  a  comparison  of  the  DoD  CPITG,  NPEG,  and  LAI  GLESAT 
attributes.  We  compared  their  processes,  how  many  questions  and  categories  were 
contained  in  the  assessments,  and  how  maturity  levels  were  defined.  We  did  not  compare 
the  NAAT  against  the  other  three  tools,  because  it  is  not  a  maturity  assessment  but  simply 
a  survey  tool  that  is  used  to  return  a  percentage  score  of  answers. 

We  compared  all  four  tools  against  the  key  concepts  described  in  Chapter  2.  All 
of  the  maturity-level  based  tools  incorporate  all  the  key  concepts.  The  NAAT  is  self- 
defined  and  developed  by  the  organization,  so  the  organization  defines  what  concepts  or 
questions  it  wants  to  ask,  so  it  can  not  be  assessed  in  the  same  way  as  the  other  three 
tools. 

As  can  be  seen  in  the  table,  all  three  of  the  maturity-level  tools  have  multi-step 
processes  within  a  defined  framework.  Both  the  GLESAT  and  the  DoD  CPITG  have 
their  origins  in  the  LAI  Transition-to-Lean  Roadmap.  The  NPEG  takes  as  its  origin  the 
Malcolm  Baldrige  Quality  Award  structure.  All  utilize  a  five-step  maturity  level 
assessment.  All  are  considered  to  be  enterprise-level  tools,  and  are  based  on  a  framework 
that  assess  the  enterprise  based  on  specific  categories,  with  synergy  in  the  topics  of 
strategic  planning,  leadership,  performance  measurement,  and  process  quality.  All 
provide  an  enterprise  self-assessment  that  is  divided  into  various  numbers  of  categories 
and  questions. 

2.  Self-Assessment  Tools  Pro's  and  Cons 

Table  9  outlines  each  of  the  tools'  pro's  and  con’s.  Analysis  was  based  on  the 
following  factors: 

•  Whether  or  not  the  tool  could  be  applied  to  an  enterprise 

•  A  solid  foundation  exists  based  on  an  industry-tested  tool 

•  It  incorporates  Lean  or  CPI  as  part  of  the  assessment 

•  Utilizes  a  maturity  level-based  approach 
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•  Ease  of  use  and  application 

•  The  requirement  for  facilitation 

•  Level  of  resources  required  for  administration 

We  did  not  apply  a  scoring  factor  to  the  tools,  but  applied  a  simple  objective 
assessment  of  each  tool. 
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Table  8.  Comparison  of  Maturity-Based  Self-Assessment  Attributes 


DOD  CPITG 

NPEG 

GLESAT 

Four-step  process,  in  a  performance 
management  framework,  developed 
from  LAI  Transition-to-Lean 

Roadmap 

Five-step  process,  incorporating 
performance  management  framework 
based  on  Malcolm  Baldrige  National 
Quality  Award 

Three  major  sections,  created  by  LAI/US AF, 
based  on  LAI  “Transition  to  Lean”  Enterprise 
Roadmap  and  LESAT 

Maturity  Assessment:  Eight 
categories,  34  questions 

Maturity  matrices:  19  Criteria,  five- 
level  maturity  matrix,  2-3  weeks  to 
complete 

Utilizes  the  Approach-Deployment- 
Learning-  Integration  technique  from 
Baldrige 

Maturity  matrices:  Three  major  sections,  54 
questions: 

•  Enterprise  Leadership, 

•  Life  cycle  and  enabling  processes 

•  Enabling  infrastructure 

Maturity  level  definitions: 

1  -  Not  yet  started 

2  -  In-process 

3  -  Demonstrated 

4  -  High  Visibility 

5  -  Transformation  Evident 

Different  self-assessment  criteria  and 
five  maturity  levels  are  defined  for  each 
section 

Maturity  levels  range  from  lean  principles  not 
evident  within  the  organization  to  lean  principles 
are  part  of  the  culture;  levels  arc  designed  for  each 
category 
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Table  8.  Comparison  of  Maturity-Based  Self-Assessment  Attributes  (cont.) 


DOD  CPITG 

NPEG 

GLESAT 

Advanced  CPI  Assessment  Tool,  based  on 

Self  assessment  categories: 

Structured  for  tracking  Entry  level 

eight  areas,  with  five-level  scoring  ranging 

•  Organizational  profile  and 

assessment  ("as  is”)  and  Re-Entry  level  (“to 

from  absence  of  the  element  to  world-class 

strategic  direction 

be"),  measured  using  summary  sheets 

implementation;  takes  45  minutes 

•  Organizational  assessment 

Categories: 

•  Strategic  planning 

Addresses  elements  of: 

•  Execution  planning 

•  Strategic  integration 

•  Leadership  Vision  and  Commitment 

•  Performance  measurement  and 

•  Leadership  and  commitment 

•  Change  Management 

results 

•  Value  stream  analysis 

•  Organizational  Structure  &  Support  system 

•  Change  management 

•  Corporate  Culture, 'Workplace  Climate 

•  Structure  and  systems 

•  Process  Stability 

•  Lean  transformation  planning,  execution 

•  Process  Quality 

•  Just-in-Time 

•  Information  Management 

Scoring  uses  a  gap  analysis,  measuring  current 
state,  future  state,  gap,  importance,  and 
evaluator  rating 

and  monitoring 
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Table  9. 


Pro’s  and  Con's  of  the  Self-Assessment  Tools 


Self- Assessment  Tool 

Pro 

Con 

DoD  Continuous  Process 
Improvement  Guidebook 
(CPI) 

•  Enterprise  level,  scalable  within  defined 
enterprise  areas 

•  Developed  from  LAI  Transition-to-Lean 

Roadmap 

•  Five-level  maturity  model 

•  Incorporates  an  performance  measurement 
model 

•  Contains  CPI  checklists 

•  Specifically  addresses  implementing  and 
measuring  Lean  implementation 

•  Addresses  key  concepts  of  knowledge 
management,  change  management 

•  Assessment  requires  facilitation 

•  Contains  a  10  question  quick  assessment, 
but  would  take  facilitated  interpretation  to 

apply 

•  Resource  intensive  to  administer 

•  Requires  “critical  mass”  of  trained  CPI 
resources 

•  Untested  within  DoD 

Navy  Performance 

Excellence  Guide  (NPEG) 

•  Enterprise  level,  scalable  to  a  defined  enterprise 

•  Five-level  maturity  model 

•  Provides  an  organizational  performance 
framework 

•  Based  on  a  proven  industry  model  (e.g., 

Baldrige) 

•  Contains  a  self-assessment,  requires  facilitation 

•  Addresses  key  concepts  of  alignment,  KM, 
change  management 

•  In  draft  form  as  of  this  writing,  so  untested 
within  Navy  except  for  limited  beta-test 

•  Could  be  resource  intensive  to  implement 

•  Leadership  may  not  have  patience  to  go 
through  all  five  steps  (could  take  over  a 
year) 

1 14 


Table  9.  Pro's  and  Con's  of  the  Self-Assessment  Tools  (cont.) 


Self-Assessment  Tool 

Pro 

Con 

LAI  GLESAT 

•  Designed  as  an  enterprise  level  self  assessment, 
part  of  a  larger  lean  enterprise  performance 
framework 

•  Thoroughly  tested  within  aerospace  industry,  and 
within  program  teams  in  US  Air  Force 

•  Provides  opportunity  for  government  and  industry 
program  teams  to  conduct  assessments  based  on 
common  framework  and  lexicon 

•  Takes  a  lot  of  training  for  participants  to 
understand  the  terminology 

•  Can  be  time-consuming  and  "painful"  to 
administer 

•  Resource  intensive  process  to  analyze  results 

•  Cumbersome  when  not  tailored  specifically 
to  the  language  of  the  organization 

NAVAIR  Alignment 
Assessment  Tool 

•  Survey  tool,  providing  easy  distribution  for  wide 
use 

•  Designed  for  web  presentation 

•  Designed  with  too  many  questions, 
categories 

•  Participants  worried  about  attribution 

•  Not  a  maturity-level  tool 
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E. 


ASSESSMENT  SUMMARY 


In  this  chapter,  we  described  the  current  NAE  environment  by  outlining  some  of 
the  issues  in  two  major  enterprise  commands.  Based  on  all  of  the  initiatives  in  place 
within  the  NAE,  it  is  possible  that  the  “signal-to-noise"  ratio  might  be  such  that  key 
messages  are  being  lost  in  all  the  activity.  Knowledge  of  the  current  environment  must  be 
understood  for  further  analysis  on  the  best  approach  for  NAE  self-assessment.  The 
aggregation  of  the  10  interviews  into  cogent  themes  provides  additional  necessary' 
information  to  reach  viable  conclusions  on  our  primary  thesis  question.  The  final  analysis 
in  this  chapter  compared  the  self-assessment  tool  attributes,  side-by-side,  with  an 
estimation  of  their  pros  and  cons.  We  now  move  to  Chapter  5,  where  we  answer  our 
research  questions  and  provide  conclusions  and  recommendations. 
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V. 


CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

In  this  chapter  we  derive  conclusions  and  recommendations  for  our  primary  and 
subsidiary  research  questions.  Based  on  our  findings,  we  suggest  areas  for  further  study 
that  fill  knowledge  gaps  for  AIR  Speed  CPI  self-assessment. 

In  Chapter  1,  we  explained  the  purpose  of  the  thesis  was  to  evaluate  four  self- 
assessment  tools  that  could  be  used  to  ascertain  the  maturity  of  the  AIRSpeed  initiative 
within  the  Naval  Aviation  Enterprise  (NAE).  The  four  tools  analyzed  were:  Department 
of  Defense  (DoD)  Continuous  Process  Improvement  Transformation  Guidebook 
(CPITG),  Navy  Performance  Evaluation  Guide  (NPEG),  Lean  Aerospace  Initiative 
Government  Lean  Enterprise  Self-Assessment  Tool  (LAI  GLESAT);  and  the  NAVAIR 
Alignment  Assessment  Tool  (NAAT).  Our  primary  question  w'as:  which  self-assessment 
tool(s)  and  methodology  can  best  capture  the  crucial  aspect  of  measuring  continuous 
improvement  in  the  NAE?  We  also  addressed  the  following  subsidiary  questions:  I  low- 
can  the  NAE  leverage  work  being  accomplished  by  the  Massachusetts  Institute  of 
Technology  (MIT)  LAI.  I  low  can  our  conclusions  be  used  to  assist  the  LAI  in  their  lean 
enterprise  transformation?  To  what  extent  can  one  tool  be  scaled  for  use  across  the 
NAE? 

In  Chapter  2,  we  provided  a  literature  review  and  defined  key  concepts  applicable 
to  the  field  of  organizational  performance  measurement,  including  tools  and 
methodologies  relevant  to  emerging  Defense  efforts  for  enterprise  transformation.  We 
explained  the  potential  value  and  benefits  of  organizational  assessment,  specifically  in 
terms  of  evaluating  how  an  organization  aligns  its  performance  initiatives  and  operations 
w  ith  its  intended  results,  i.e.,  culture,  outputs  and  outcomes.  Are  we  accomplishing  our 
goals,  and  obtaining  desired  or  intended  consequences? 

In  Chapter  3,  we  provided  an  overview  of  each  of  the  four  self-assessment  tools, 
including  a  discussion  of  various  structural  and  process  elements.  In  Chapter  4,  we 
analyzed  the  relative  advantages  and  disadvantages  of  each  tool,  including  alignment 
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with  incremental  and  transformational  concepts  and  improvements.  We  provided  the 
results  of  interviews  conducted  with  senior  and  middle  managers  within  major  aerospace 
companies,  and  civilian  and  active  duty  personnel  in  the  U.S.  Navy  and  U.  S.  Air  Force. 

B.  PRIMARY  RESEARCH  QUESTION 

Which  self-assessment  tool(s)  and  methodology  best  captures  the  crucial  aspect  of 
measuring  progress  of  continuous  process  improvement  within  the  NAE? 

1.  Conclusion 

Three  of  the  four  assessment  tools  have  applicability  in  terms  of  being  able  to 
reasonably  and  efficiently  measure  AIRSpeed  maturity:  the  Department  of  Defense 
Continuous  Process  Improvement  Transformation  Guidebook  (DoD  CPITG),  Navy 
Performance  Excellence  Guidebook  (NPEG),  or  the  Lean  Aerospace  Initiatives' 
(LAI)  Government  Lean  Enterprise  Self-Assessment  Tool  (GLESAT). 

All  of  the  tools  (except  the  NAAT)  are  based  on  a  maturity  level  construct  and  a 
performance  management  framework,  and  all  are  modeled  after  world-class  industry  self- 
assessment  tools  such  as  Malcolm  Baldrige  Quality  Program  or  Capability  Maturity 
Model  Integration  (CMMI).  They  all  can  be  used  to  measure  enterprise  progress  towards 
higher  performance,  including  "lean"  and  transformative  maturity. 

Two  conclusions  drawn  from  interviews  are  noteworthy: 

(1)  The  purpose  of  an  assessment  can  be  achieved,  only  to  the  extent  to  which 
it  is  effectively  communicated  to  the  workforce;  and 

(2)  Trust  in  the  intent  of  the  assessment  process  appears  to  be  crucial  for  mid¬ 
level  managers  and  employees,  i.e.,  personnel  need  assurances  that  any 
actual  or  perceived  negative  findings  will  be  treated  as  opportunities  for 
improvement. 

To  the  extent  to  which  distrust  and  doubt  surrounding  findings  are  present,  the 
assessment  tool  becomes  suspect  as  participants  may  skew  inputs  to  maintain  positive 
outcomes.  Obviously,  these  types  of  behaviors  and  coping  mechanisms  would  be 
detrimental  to  serious  or  sustained  performance  improvements.  This  finding  crosses  over 
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into  aspects  of  organizational  culture,  which  includes  how  the  organization  responds  to 
defects,  errors,  and  even  downsizing  that  may  be  associated  with  cost-cutting  initiatives. 
Therefore,  performance  assessment  or  use  of  an  assessment  tool  can  depend  on 
organizational  climate  and  culture,  including  positive  and  negative  reward  processes. 
Obviously,  a  decline  in  organizational  morale  would  be  an  unintended  consequence  of 
not  understanding  the  importance  of  communications  and  intent  concerning 
organizational  assessment. 

Summarizing  conclusions  derived  from  the  literature,  organizations  must  align 
their  measurement  systems  to  their  strategic  intent  (Ancona,  2005).  Changing  a  culture  is 
complex,  and  a  new  culture  emerges  when  there  is  “fit"  among  key  organizational 
variables  such  as  strategy,  structure,  and  results.  All  of  the  organizational  entities  need  to 
align  their  measurement  activities  for  overall  “fit,"  i.e.,  with  their  communications  and 
awards  system.  These  findings  can  be  generalized  whether  it  is  business  performance, 
“lean"  initiatives,  or  strategic  transformative  goals  and  initiatives. 

Another  related  conclusion  (gleaned  primarily  from  our  interviews)  is  that  the 
information  and  knowledge  gained  from  a  self-assessment  process  must  be 
communicated  expeditiously  to  participants  to  maximize  the  benefits.  Specifically,  by 
receiving  timely  feedback,  actions  can  be  better  applied  to  strategic  planning  and  budget 
formulation  efforts.  Waiting  for  longer-term  results  may  cause  involved  personnel  to 
forget  and/or  postpone  usable,  relevant  knowledge.  Quite  simply,  leaders  and  managers 
would  want  to  build-in  short-term  and  medium-term  visible  improvements  to  maximize 
the  intended  positive  effects  on  a  participating  workforce. 

2.  Recommendation 

a.  Establish  an  Organizational  Performance  Framework 
An  organizational  performance  or  management  system  framework  (such 
as  defined  in  the  Malcolm  Baldrige  Quality  Award  or  the  NPEG)  should  be  established 
by  the  NAE  before  any  formal  assessment  process  is  implemented.  The  rationale  would 
be  to  align  all  NAE  entities  alongside  a  single  strategic  tool  and  process  containing 
cascading  goals  and  objectives. 


1 19 


In  the  near  term,  each  NAE  Echelon  III  Command  (Wing  and  above. 
Warfare  Centers,  etc.)  would  complete  the  NPEG  Step  1  (Organizational  Profile)  and 
Step  2  (Organizational  Assessment,  minus  cost).  The  NAE  would  benefit  by  applying  a 
performance  management  framework  across  its  system,  which  would  identify  short  and 
longer-term  improvements,  as  they  become  apparent.  Knowledge  management  concepts 
and  ways  to  promote  organizational  learning  (Senge,  1990)  could  be  simultaneously 
incorporated  to  capture  maximum  value. 

By  completing  Steps  1  and  2  of  the  NPEG,  the  NAE  would  have  an  initial 
self-assessment  of  their  entire  enterprise  against  world-class  criteria  and  show  alignment 
with  SECNAV  and  CNO  guidance  and  goals.  NAE  leadership  will  be  able  to  see  the 
enterprise  as  a  “system"  and  have  useful  indicators  for  alignment  to  customers, 
stakeholders,  and  strategic  goals.  In  the  long  term,  all  NAE  commands  could  complete 
the  five-step  NPEG  Performance  Excellence  process.  The  deliverables  include 
command-generated  and  aligned  strategic  plans  and  execution  plans,  and  a  complete  self- 
assessment  against  baseline  data. 

b.  Utilize  LAI  GLESA  T for  Lean  Organizations 

The  NAE  functional  areas  that  have  adopted  lean  principles,  with  visible 
and  validated  results,  should  apply  the  LAI  GLESAT  tool  within  their  “enterprise,"  using 
the  appropriate  facilitation  (e.g.,  LAI  facilitator  or  trained  Black  Belt). 

Assessments  should  be  performed  at  least  once  a  year  for  three  years,  to 
enable  visible  progress  through  maturity  levels,  based  on  quality  data.  Additionally,  by 
implementing  an  assessment  across  multiple  business  cycles,  efficiencies  can  be  gained 
based  on  a  learning  curve  theory  for  the  participants. 

c.  Implement  CM  MI  for  RDT&E  Organizations 

In  addition  to  NPEG  and  GLESAT  for  the  larger  enterprise,  CMMI  could 
be  effectively  implemented  within  the  NAE  research  and  development,  test  and 
evaluation  (RDT&E)  organizations  across  “like"  functions.  CMMI  integrates  systems 
engineering,  software  engineering  and  acquisition,  and  integrated  product  and  process 
development.  The  use  of  CMMI  within  the  RDT&E  organizations  would  provide  an 
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extensible  framework  and  an  evolutionary  path  towards  improvement  through  the 
maturity  levels.  It  would  provide  mature,  documented  processes  across  the  NAE  within 
the  technical  disciplines. 

d.  Annual  Use  ofNAA  T  for  the  NAE 

We  recommend  that  NAE  use  the  NAAT  yearly,  but  the  number  of 
questions  and  categories  should  be  reduced  based  on  feedback  received  after  the  last 
survey  from  employees  and  managers.  Also,  interviews  with  NAVAIR  personnel 
revealed  that  having  to  manually  organize  the  comments  submitted  took  a  number  of 
days.  Utilizing  a  restructured  tool,  method,  or  survey  that  could  quickly  translate  written 
comments  to  common  threads,  would  be  more  valuable  in  providing  timely  and 
actionable  information. 

e.  Resource  Commitment  ami  Change  Agents 

One  of  the  criteria  for  analyzing  the  four  tools  was  the  amount  of 
resources  required  to  administer  the  tool  within  an  organization.  Organizational 
assessment  with  any  of  the  tools  demands  substantial  resources:  facilitators,  managers, 
program  team  members,  time,  and  data  analysis.  With  the  current  environment  of  excess 
organizational  “noise"  where  essential  messages  or  “signals"  can  become  lost  within  the 
NAE,  it  is  doubtful  that  applying  a  complex,  total  enterprise  self-assessment  could  be 
successfully  accomplished,  due  to  the  level  of  support  required  to  educate  and  prepare 
NAE  personnel  to  a  point  where  useful  information  could  be  collected.  We  recommend 
that  any  self-assessments  be  targeted  to  “high  value  targets"  identified  through  the  NPEG 
Step  1  and  2. 

Dedicated  local  change  agents  that  could  administer  organizational  self- 
assessment  could  provide  the  return  on  investment  to  ensure  acceptance  and  focus.  Each 
tool  has  a  level  of  complexity  that  docs  not  lend  itself  to  just  being  given  to  groups  or 
employees  and  having  them  accepted  and  used.  The  tools  are  powerful  when 
administered  by  a  trained  user,  such  as  a  LAI  facilitator.  An  internal  change  agent,  such 
as  one  of  the  AIR Speed  Black  Belts,  after  the  appropriate  familiarization  and  training. 
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would  be  a  good  choice  to  administer  any  of  the  maturity-level  tools  within  a  particular 
program  or  organization. 

C.  FIRST  SUBSIDIARY  QUESTION 

How  can  the  NAE  leverage  work  being  accomplished  by  the  MIT  LAI? 

I.  Conclusion 

LAI  is  an  active  consortium  that  brings  together  major  aerospace  companies  and 
government  to  share,  enhance,  and  improve  their  CPI  efforts.  There  are  many  ways  that 
the  NAE  can  leverage  the  work  the  LAI  is  doing.  LAI  has  created  a  toolset  (which 
includes  the  GLESAT)  that  is  designed  for  enterprise  lean  transformation.  LAI  has  an 
active  research  program,  which  is  currently  in  its  fifth  phase,  which  has  three  key  drivers: 
1 )  the  need  to  accelerate  lean  enterprise  transformation;  2)  the  need  to  design  future  lean 
enterprises;  and  3)  the  need  to  evolve  adaptive  lean  enterprises.  To  support  these  key 
drivers,  the  current  phase  of  research  has  four  core  questions: 

•  How  can  1  understand  how  my  organization/enterprise  currently  operates 
within  its  larger  context? 

•  How  can  I  define  and  evaluate  the  future  possibilities  for  evolving  a  more 
efficient  enterprise? 

•  What  are  the  most  effective  strategies  and  tactics  to  achieve  these  future 
possibilities  for  my  enterprise? 

•  1  low  can  1  best  manage  the  enterprise  change  process? 

In  addition,  the  yearly  LAI  Plenary  Conference  provides  opportunity  for  building 
relationships  with  members  and  taking  advantage  of  knowledge  sharing.  There  are  a 
number  of  MIT  students  that  present  their  work  in  progress  on  lean  topics,  as  well  as 
opportunities  to  have  one  of  those  students  study  NAE  processes  and  provide  their 
knowledge  to  help  NAE  grow  and  mature  in  lean  efforts. 
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2.  Recommendation 

As  explained  above,  AIR Speed  could  utilize  the  GLESAT  within  those  functional 
areas  that  have  visible  ‘'lean”  results.  LAI  could  assist  and  train  a  Black  Belt  to  lead  the 
assessment.  The  Air  Force  has  effectively  implemented  LESAT  and  it  GLESAT  version, 
across  the  C-17  enterprise.  The  use  of  a  common  assessment  tool  as  part  of  the  overall 
MIT  LAI  enterprise  transformation  strategy  using  the  Transition  to  Lean  (TTL)  and 
Enterprise  Value  Stream  Mapping  (EVSMA)  helped  the  C-17  to  create  a  common  view 
of  the  enterprise,  with  actionable  data  to  fix  gaps  between  the  current  state  and  the 
desired  state  in  the  program. 

Participating  in  the  yearly  LAI  Plenary  Conference  would  provide  NAE  personnel 
with  the  opportunity  to  meet  and  talk  with  many  members  of  leading  aerospace 
companies  and  share  success  stories  and  experiences  of  their  “lean"  journeys.  We  found 
those  contacts  essential  when  we  conducted  our  interviews.  We  also  utilized  the  results 
of  some  of  MIT's  doctoral  and  master's  research  studies  in  enterprise  lean  transformation 
in  our  work.  We  will  be  sharing  the  results  of  this  thesis  work  with  LAI,  which 
strengthens  the  NAVA1R  and  NAE  relationship  with  them. 

D.  SECOND  SUBSIDIARY  QUESTION 

How  can  our  learning  be  translated  to  provide  value  to  the  MIT  LAI  research  in 
lean  enterprise  transformation? 

1.  Conclusion 

We  have  shown  during  our  analysis  that  the  use  of  the  LESAT  and  GLESAT  tools 
has  helped  both  industry  and  government  focus  their  lean  efforts  and  determine  the 
maturity  of  their  progress.  During  the  research  phase  of  our  thesis,  we  conferred  with  a 
number  of  master’s  and  doctoral  MIT  students  who  were  doing  research  in  the  lean 
enterprise  transformation  area.  We  also  discussed  our  findings  with  regard  to  the  LESAT 
tool  with  LAI.  During  this  year’s  Plenary  Conference,  we  met  many  MIT  students  who 
were  looking  for  further  opportunities  to  interface  with  industry  and  government  in  order 
to  complete  their  theses. 
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2.  Recommendation 

We  believe  that  the  NAE  should  encourage  additional  personnel  pursuing 
graduate  education  studies  (theses  or  group  projects)  in  the  area  of  Lean  and  enterprise 
transformation.  Those  efforts  could  augment  the  current  research  being  conducted  by  the 
LAI. 

Students  in  the  Product  Development  for  the  21s'  Century  (PD-21)  Program  at  the 
Naval  Postgraduate  School  could  do  follow-on  research  in  the  LAI  lean  enterprise 
transformation  areas. 

MIT  students  who  are  doing  research  in  “lean”  transformation  could  be  invited  to 
do  a  “tour”  or  short  assignment  within  NAE  to  share  their  knowledge  and  gain  new 
knowledge  in  AIRSpeed  accomplishments  in  lean. 

Our  recommendation  to  LAI  is  that  they  form  a  LESAT/GLESAT  User's  Group 
so  that  those  companies  and  government  activities  that  have  utilized  the  tool  could 
provide  feedback  and  learning  to  help  improve  the  tool. 

E.  THIRD  SUBSIDIARY  QUESTION 

Can  a  single  tool  be  scaled  for  use  across  the  NAE? 

I.  Conclusion 

Our  research  has  shown  that  assessment  tools  by  themselves  are  not  the  panacea 
to  achieving  the  desired  cultural  change  to  a  lean  or  transformed  enterprise.  Senior 
leaders  and  managers  are  influenced  to  view  their  organizations  as  systems  of 
interdependent  variables  working  toward  a  common  purpose,  as  such,  assessment 
interventions  must  be  aligned  with  other  key  organizational  variables;  coupled  with 
tremendous  effort  to  ensure  that  work  design  improvements  match  customer  requirements 
and  internal  strategy.  The  cognitive  act  of  “choosing”  a  tool  and  process  must  be 
followed-through  with  the  more  mundane  aspects  of  clear,  consistent  communications 
and  continuity  across  commanders'  intent. 

The  NAE  has  various  organizational  functions  and  structures.  As  a  result  of 
analyses  and  interviews,  “enterprises”  are  defined  as  a  single  program  office  or  a 
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complete  organization.  A  single  tool  such  as  the  NPEG  could  be  effectively  scaled  within 
the  NAE;  it  was  designed  with  the  broad  definition  of  “enterprise”  in  mind.  Also,  the 
DoD  CPITG  and  the  GLESAT  were  designed  to  be  used  by  and  within  enterprises,  and 
they  could  be  scaled  for  use  as  well. 

2.  Recommendation 

Using  our  Assessment  Key  Concepts,  proactively  determine  the  best  fit  for  self- 
assessment  tools  by  functional  business  area.  Once  this  is  done,  the  process  and  tool  can 
scale  to  the  targeted  business  area  within  the  NAE.  Using  the  NPEG  would  be  the  best 
way  to  start  the  assessment  process  for  AIR Speed. 

F.  AREAS  FOR  FURTHER  RESEARCH 

The  following  questions  could  be  answered  by  future  research: 

1 .  What  is  the  maturity  level  of  MRSpeed  across  the  NAE?  By  applying  the 
GLESAT  tool  within  select  areas  that  have  been  on  the  “lean  journey"  for  a  number  of 
years,  a  preliminary  level  of  maturity  could  be  obtained  as  well  as  information  for 
organizational  learning. 

2.  Can  the  principles  of  High  Performance  Organizations  (I IPO)  provide 
additional  value  to  MRSpeed?  I  IPO  has  been  widely  used  within  the  NAVAIR 
community,  specifically  within  the  software  engineering  department  and  Depots.  HPO 
concepts  would  seem  to  provide  a  level  of  the  performance  management  framework  that 
could  be  of  value  to  NAE  and  help  shape  positive  behavior  of  leadership. 

3.  During  the  interviews,  we  discovered  what  seemed  to  be  the  two  most 
common  organizational  barriers  to  true  enterprise  behavior:  a  rigid  budget-focused 
business  planning  process  and  misaligned  management  award  and  reward  policies.  This 
observation  was  consistent  across  government  and  industry. 

The  budget-focused  organization  can  be  the  most  obstructive  to  enterprise 
transformation  because  the  barriers  to  enterprise  thinking  are  embedded  into  the  daily 
management  and  tracking  processes  at  the  business  unit  level.  Managers  define  success 
by  obtaining  funding  and  protecting  budgets.  Because  their  decisions  are  not  focused  on 
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overall  enterprise  metrics,  they  sub-optimize  decisions  for  local  benefit.  In  some  cases, 
performance  awards  are  designed  to  support  this  behavior. 

To  begin  transformation  from  a  budget- focused  culture  the  NAE  comptroller 
organizations  must  be  fully  integrated  into  the  NAE  CPI  process.  They  must  be  included 
or  knowledgeable  of  all  the  non-financial  decisions  and  design  of  enterprise  work. 

We  recommend  that  the  NAE  use  Lean  Six  Sigma  tools  to  review  the  budget 
structure  and  processes  to  define  the  barriers  to  positive  enterprise  behavior.  This  review- 
should  not  be  lost  in  detail  and  should  stay  at  the  macro  process  level  -  above  numbers  -- 
where  behaviors,  attitudes,  and  accepted  counter-enterprise  practices  can  be  identified  for 
removal  by  senior  staffs. 
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